[tor-dev] How many exits exit from an IP address different than their OR address? (10.7%)

Tue Jan 12 03:43:29 UTC 2016

I wanted to know how many exits exit from an address that is different
from their OR address. The answer is about 10.7%, 109/1018 exits. The
interesting part is that of those 109 mismatches, 87 have an exit
address that differs from the OR address in all four octets; i.e., the
IP addresses used by the exit are not even in the same /8.

Also, there are several groups of exits whose OR addresses *are* in a
related subnet, but which all exit through the same, unrelated, IP
address. See for example 109.236.82.* (all exiting through
185.108.128.7), 178.17.171.* (all exiting through 37.48.65.71), and
217.23.11.* (all exiting through 85.17.31.120) in the table below.

I wrote an exitmap module that hits an IP-checker website. The module
wrote a CSV file where each row included the exit's OR address and the
address as reported by the remote web site. I ignored IPv6 OR addresses.
I did not find any fingerprint that had more than one IPv4 address
(considering the .fingerprint and .or_addresses members from Stem).
Percentages are just raw counts, not weighted by exit probability.

Total number of exits: 1018 (100.0%)
Number of failures: 18 (1.8%)
Number of matches: 891 (87.5%)
Total number of mismatches: 109 (10.7%)
Number of 1-octet mismatches: 19 (1.9%)
Number of 2-octet mismatches: 3 (0.3%)
Number of 3-octet mismatches: 0 (0.0%)
Number of 4-octet mismatches: 87 (8.5%)

Table of mismatches
   * = mismatch 1 octet
  ** = mismatch 2 octets
 *** = mismatch 3 octets
**** = mismatch 4 octets
                              fingerprint      or_address  remote_address mark
 B060482C784788B8A564DECD904E14CB305C8B38  176.10.104.241  176.10.104.240    *
 DC41244B158D1420C98C66F7B5E569C09DCE98FE  176.10.104.241  176.10.104.240    *
 487092BA36F4675F2312AA09AC0393D85DAD6145  176.10.104.244  176.10.104.243    *
 DE7DE889E0D1A5F397AE35642060B84999581203  176.10.104.244  176.10.104.243    *
 7A65CC8F45134F3393A0295EFFD2980DD885E8E2   176.123.6.157   176.123.6.155    *
 A7E78D9880BB0793409D386D15E83B3B1236B19F  178.175.128.54  178.175.128.50    *
 8BF22BEA5F854F2BB2E09C71260ADF590D355E36  213.61.149.125  213.61.149.100    *
 80D73E75A30BEEF627604B7014753304764E0723  213.61.149.126  213.61.149.100    *
 5D263037FC175596B3A344132B0B755EB8FB1D1C   31.185.27.203     31.185.27.1    *
 C974508A98446F36169FB248655BCD50DF17F14C  37.130.227.134  37.130.227.133    *
 170EC06D58E9094A027F4169514ADD98D30983A6   46.235.226.27  46.235.226.226    *
 D3BE73ECE50DA9243DC0DF9DA6ED25027B82D385    50.7.178.101    50.7.178.100    *
 FEDEF82551BB49DB54BE6C77D27AD0E20A8D6FD6    50.7.178.102    50.7.178.100    *
 10E13E340651D0EF66B4DEBF610B3C0981168107  77.247.181.164  77.247.181.162    *
 06E123865C590189B3181114F23F0F13A7BC0E69  77.247.181.166  77.247.181.162    *
 664BDC0344771122FC6C4F577BF0AEC3F4ED5456   80.73.242.142   80.73.242.130    *
 9E3E47D7B92144CBAB0D487CCE192531F6A55833   81.209.35.111   81.209.35.112    *
 410C286061266C562049178C6BF1E04060F51F7C   94.103.175.85   94.103.175.86    *
 F4B72EA6FD0EACF652B6C200611F37244F2B31F3  94.242.228.107  94.242.228.108    *
 62BEE61EB88D4A81C3BA3931D6FA999D706AC4D5    62.149.13.57   62.149.12.153   **
 1BC17B72F90998F0BAEE25E36FFC140C9D7A8D7A     85.119.82.4    85.119.84.57   **
 AA60D088E8317BBA3D2CF96C706AB4452FB2F57D  90.182.128.100   90.182.235.46   **
 4E915A5CCDBBDD4BD4B7F09A5AFB581AB1AD7EF2 107.181.178.204  104.156.228.72 ****
 03B4386E579BEBCF7605D0FB17A688B35C342D5D   109.236.82.50   185.108.128.7 ****
 C3B1E12BCA2F307B64737AB84993E7B414FD3D09   109.236.82.50   185.108.128.7 ****
 B9DC592CBC10EC7428730BF658F9211B8773800F   109.236.82.51   185.108.128.7 ****
 DE89B3B6954FFC0F76F6B37A05E2AD6EF3B046F1   109.236.82.51   185.108.128.7 ****
 CF9F5B88C0F95EB6C5547CE5FB38503049DF784D   109.236.82.52   185.108.128.7 ****
 57275A9356CFB1D36E64B117D44AB9F67711E483   109.236.82.53   185.108.128.7 ****
 D1E52741A7DCB4D51C7EFC82A654E2527516E1B0   109.236.82.53   185.108.128.7 ****
 7837F5B740223D19D8A0D3127D2A8EB9FDA59F67  148.163.73.112 206.217.208.162 ****
 B6045286F21E7078203E1B03C8E00669A2154724  148.163.73.112 206.217.208.162 ****
 D225C2EC81E044E3947BC49929E46003ED62C89F  151.80.164.147  46.166.186.238 ****
 23A69F271ACBFACB20467559D1C04F13DEF74B4B  153.92.126.135  128.127.105.94 ****
 4236DF494DC91E1245234AA676F1F993ECB067F5  153.92.126.135  128.127.105.94 ****
 27CE487BABE6C2128FB2D1A801C7CB715B48EC6B   153.92.126.19      168.1.6.34 ****
 C369EB6BFDFB996864746944544B82C9684C46EA   153.92.126.19      168.1.6.34 ****
 5453CAF677B8E754C4D5EFA1FEDD8ABB16950FCB  153.92.127.143    168.1.10.226 ****
 756F68024C6E55FF7142AD933C135F88B552A932  153.92.127.143    168.1.10.226 ****
 B0E7C83FA0C728997BB60CB0C758A3A3FDC3C1DB 173.243.112.148  162.216.46.165 ****
 E6BE2BEAAF4A543F9D94CD20CB459ECFEA3AD1B1 173.243.112.148  162.216.46.165 ****
 7585DDCEB14783C69960E777BA258F5B3948F1C9  176.126.85.175 159.122.133.198 ****
 CBBCD92AD9479C8FBB8AA17FF22C7EC206FF5B1A  176.126.85.175 159.122.133.198 ****
 9634A0F1FBE02EBF6E0D59CA7835928D6D55CBCA  176.126.85.176  161.202.72.152 ****
 C68988E3DD72549AD9C3D1A67FAD056241154775  176.126.85.176  161.202.72.152 ****
 58287449DDAD3CF4020071E1139CB4A22BB02D02  178.17.171.140     37.48.65.71 ****
 7E2AFD5EA1FD32B0EBA4BDC3C102B9DC67B882E7  178.17.171.140     37.48.65.71 ****
 59DE28A0D0A5529CEA9FC4B90DFCFCEC980C4139  178.17.171.156     37.48.65.71 ****
 E94FA1FDEDF7FC10208C51DABD4EC5045C0A4EC4  178.17.171.156     37.48.65.71 ****
 52F6510B5D59678B77DF351C0F7F9B5A2070F393  178.17.171.157     37.48.65.71 ****
 A566EBAC657485F3D0AEE70C021EA2658D9F0B09  178.17.171.157     37.48.65.71 ****
 BCF9B6FA788E7F74DD77724D205C46195688D748  178.17.171.159     37.48.65.71 ****
 D41C2006D7C461BDC22B9D236CC93F5D366C1388  178.17.171.159     37.48.65.71 ****
 5CECC5C30ACC4B3DE462792323967087CC53D947   178.32.181.96  37.187.129.166 ****
 65F9944338C684109EB975D0EC7489B30E191E87   178.32.181.97  37.187.129.166 ****
 66AA1EB64AAFEDC1EA8E49A701F6C472102C5E1A   178.32.181.98  37.187.129.166 ****
 4F0003EAB0E8712B94B29A84BF8B8F34C95927BC 185.106.120.153 216.185.103.139 ****
 4287D0C57CEFBE7E06B24A0E9370DC671FED5462   185.62.190.38    93.115.87.78 ****
 D96F409E59D5C41F3761F1F23438131BDE57C538   185.62.190.38    93.115.87.78 ****
 8E15D869D266DF5B08351122185F9660CA6D0FA2  185.82.202.178 216.185.103.139 ****
 BF6379111EC2C29C104ED0239D5709D08BC3B9EB  188.209.52.109    5.149.249.10 ****
 184E9215A97F21323BF8661329FCB6F89305CDAC    192.3.24.227 173.225.119.156 ****
 9AC41D1DCA4A1BB5D52708E9EE00CED129567B7C    192.3.24.227 173.225.119.156 ****
 7DF1BD29A7F927B61F31688395D39F75FC541647   209.141.43.84  67.212.234.181 ****
 86D3FECC084BE5611B3F7F4791DAB6204C2E2996   209.141.43.84  67.212.234.181 ****
 5F2F5B0014BE0C36E98EEAD5FC35A100FCCEDD4B    217.23.11.95    85.17.31.120 ****
 CC8C96DF58EFC67662FF751425D2A4D6C7D0F8F0    217.23.11.95    85.17.31.120 ****
 1BE59B741F6DAECF29F13226C33B3282316BD81C    217.23.11.97    85.17.31.120 ****
 83FCB9E1BE894C3947686AB6C11074B4AAA959B7    217.23.11.97    85.17.31.120 ****
 AB802EDBD1CF49CDE5A03394AC1B20C5DC1E17CE    217.23.11.98    85.17.31.120 ****
 C8D9AD239C658121999E56196A8AF74FF586A073    217.23.11.98    85.17.31.120 ****
 A478E6F4F193BDE43C3BD2BC6A5BC90DA5472FE8     217.23.7.25    168.1.99.217 ****
 B3233E96418C4E89AF22FC9A1F2392C0D194117B     217.23.7.25    168.1.99.217 ****
 4AF6CEC4351DE02BB5A7CBBC82607171832C687B     217.23.7.79    168.1.99.217 ****
 26ABA62FBEE9A751790C31AA9EE9E5014439769C     217.23.7.98    168.1.99.217 ****
 382248E9E99857109B14344CF2B88166B41297A3     217.23.7.98    168.1.99.217 ****
 137AEDE28D9E0BBABE2A5B2F476096740EF1A226     217.23.7.99    168.1.99.217 ****
 69ABCEEE7117BBBBF74BA03C09963D28A4F7376F     217.23.7.99    168.1.99.217 ****
 FA4E22CFC0802D6738837DE980AFC7199F7DB040  217.34.135.225     136.0.2.226 ****
 68F70BFCB2A5400ED6F98C8DB30C41A36743962D   217.37.19.115  95.211.184.197 ****
 C9A7289A94FF30C8772D7E70B56EACA330B95D08   31.220.42.170  95.211.148.154 ****
 D8082DFB5E22F23D60DE1F6A80986323FF86C027   31.220.42.170  95.211.148.154 ****
 D90ED0B491FB089EEA827188891BB47EBE13DC14   37.48.124.116    85.17.31.120 ****
 276A7ACA7515E90ABB2AB36D4F3421B1C7FB4501   37.48.124.117    85.17.31.120 ****
 5FEA88EFEDC5897F7094A1759CE25BA0D3DA87E7   37.48.124.117    85.17.31.120 ****
 1F85C11528B6C599EDC488A68DF045F73D29CB18     37.48.64.48  109.201.143.40 ****
 F712F85AFA84A4D68A40A7AC20416D0B426969CD     37.48.64.48  109.201.143.40 ****
 0C7A04579C0A53D99389333D2F9BAE35589F0F84     37.48.64.49  109.201.143.40 ****
 71AE6B66F521BD5E6F304A08FB2ABADF4557D815     37.48.64.49  109.201.143.40 ****
 B9C49DBE93D7D70B41CF95C4F91EFDFC73FC5E60  46.105.183.141   108.61.123.75 ****
 D3DA6122AA416B8D7AB02CA9858DC3F206C656D2  46.105.183.142   108.61.123.75 ****
 D85E3FFE70BCBC6964969CDF1065457736D99DE4    50.7.124.238  161.202.72.185 ****
 1AE0B173A9FA95D4258B9FC181FDB2BBF040F913    50.7.124.243  161.202.72.185 ****
 C5325DED0CFEA73F381920FADF5F1220B72841B8    50.7.124.243  161.202.72.185 ****
 68D9005982673F068B7844D60915FD141155A8AC   51.254.83.238   108.61.123.75 ****
 7774C0F736E02AD48C7A4F3BDFE636A21BBE0255   51.254.83.238   108.61.123.75 ****
 4F154FF308400D5BDFB791FAD4C7577547BAA5CA  74.122.198.101      168.1.6.28 ****
 C75EC9CB9667CEC47BB155DD322789DBE0384ED8  74.122.198.101      168.1.6.28 ****
 308D5B726051761D40DB31C7E89254A3CD3F27E0    78.142.19.59     85.17.25.22 ****
 3D28E5FBD0C670C004E59D6CFDE7305BC8948FA8    78.142.19.59     85.17.25.22 ****
 70119F72F2609BA1CC12640BF66905FE21768152   79.143.87.204 104.238.169.126 ****
 83D664A8216DBE4BD48F71BDC61E8F7D8AB68517   79.143.87.204 104.238.169.126 ****
 C7E75D7D968C2355791EC29A420F69AF2737DA58  93.158.215.174     31.7.56.133 ****
 0FAE6840763B2C9DDBCDD9DBA7C1A0549FF0155A  94.242.206.183   93.115.83.253 ****
 13DFA359319F34F3FFC55154BDBC499FB5822C1C  94.242.206.183   93.115.83.253 ****
 0760C0A34CCBCB2614075E27B88151FCC7D047EC   94.242.206.35   93.115.83.253 ****
 BFD74D651690324DCA9FB744E2D54DDC814DD3FB   94.242.206.35   93.115.83.253 ****
 0BE8EBA01005974782C4573037F4F82D915166C5    96.43.142.28   80.82.215.199 ****
 A9FEB4AD16FC0ECCF5B01885374E4243E0981384    96.43.142.28   80.82.215.199 ****
-------------- next part --------------
A non-text attachment was scrubbed...
Name: exit_addr.csv.gz
Type: application/gzip
Size: 35623 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20160111/3e331909/attachment-0001.bin>
-------------- next part --------------
failed <- function(row) {
	row$remote_address == ""
}

# Return a string in "", "*", "**", "***", "****" signifying how many
# contiguous octets don't match starting from the right.
mark <- Vectorize(function(a, b) {
	ifelse(b == "", "", paste(rep("*", sum(!Reduce(`&`, unlist(strsplit(a, ".", fixed=T)) == unlist(strsplit(b, ".", fixed=T)), accumulate=T))), collapse=""))
})

options(width=120)

x <- read.csv("exit_addr.csv", colClasses=c(fingerprint="factor", or_address="character", remote_address="character"))

# Ignore IPv6.
x <- x[!grepl(":", x$or_address), ]

# Ensure there are not multiple IPv4 addresses for any fingerprint.
if (any(duplicated(x$fingerprint))) {
	stop()
}

# Mark with mismatches.
x$mark <- mark(x$or_address, x$remote_address)

total <- nrow(x)
cat(sprintf("Total number of exits: %d (%.1f%%)\n", total, 100*total/total))

x.failures <- x[failed(x), ]
num.failures <- nrow(x.failures)
cat(sprintf("Number of failures: %d (%.1f%%)\n", num.failures, 100*num.failures/total))

x.match <- x[!failed(x) & x$mark == "", ]
num.match <- nrow(x.match)
cat(sprintf("Number of matches: %d (%.1f%%)\n", num.match, 100*num.match/total))

x.mismatch <- x[!failed(x) & x$mark != "", ]
num.mismatch <- nrow(x.mismatch)
cat(sprintf("Total number of mismatches: %d (%.1f%%)\n", num.mismatch, 100*num.mismatch/total))

x.mismatch1 <- x[!failed(x) & x$mark == "*", ]
num.mismatch1 <- nrow(x.mismatch1)
cat(sprintf("Number of 1-octet mismatches: %d (%.1f%%)\n", num.mismatch1, 100*num.mismatch1/total))

x.mismatch2 <- x[!failed(x) & x$mark == "**", ]
num.mismatch2 <- nrow(x.mismatch2)
cat(sprintf("Number of 2-octet mismatches: %d (%.1f%%)\n", num.mismatch2, 100*num.mismatch2/total))

x.mismatch3 <- x[!failed(x) & x$mark == "***", ]
num.mismatch3 <- nrow(x.mismatch3)
cat(sprintf("Number of 3-octet mismatches: %d (%.1f%%)\n", num.mismatch3, 100*num.mismatch3/total))

x.mismatch4 <- x[!failed(x) & x$mark == "****", ]
num.mismatch4 <- nrow(x.mismatch4)
cat(sprintf("Number of 4-octet mismatches: %d (%.1f%%)\n", num.mismatch4, 100*num.mismatch4/total))

cat("\n")
cat("Table of mismatches\n")
cat("   * = mismatch 1 octet\n")
cat("  ** = mismatch 2 octets\n")
cat(" *** = mismatch 3 octets\n")
cat("**** = mismatch 4 octets\n")
print(x.mismatch[order(x.mismatch$mark, x.mismatch$or_address, x.mismatch$fingerprint), ], row.names=F)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: exit_addr.py
Type: text/x-python
Size: 1410 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20160111/3e331909/attachment-0001.py>