[tor-dev] What does TorBulkExitList.py return?

[Arlo Breault]

> On Mar 17, 2016, at 3:45 AM, Linus Nordberg <linus at torproject.org> wrote:
> 
> Hi,
> 
> I'm trying to figure out why a list from [TorBulkExitList.py] is so much
> larger than what is seen in [exit-addresses].

Let's certainly not rule out the possibility of a bug.


> Point in case: Earlier today the list from TorBulkExitList.py contained
> 58% more addresses than exit-addresses:
> 
> --8<---------------cut here---------------start------------->8---
> $ curl-tor -q 'https://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=8.8.8.8' | egrep -v ^# | wc -l
> 1554
> $ curl-tor -q https://check.torproject.org/exit-addresses | egrep ^ExitNode | wc -l
> 985

Shouldn't this grep `ExitAddress`?
(which only adds 3 right now)


> --8<---------------cut here---------------end--------------->8---
> 
> Counting relays with the Exit flag in a consensus of roughly the same
> time gives 912:
> 
> --8<---------------cut here---------------start------------->8---
> $ curl-tor -q http://171.25.193.9:443/tor/status-vote/current/consensus | egrep '^s.*Exit' | wc -l
> 912
> --8<---------------cut here---------------end--------------->8---
> 
> 
> If I read the code [check.py][exitips.py] correctly, we include the
> following IP addresses in what is served from [TorBulkExitList.py]:
> 
> - all routers in the current consensus
>  - with exit addresses from TorDNSEL file(s) added
> - for which stem's exit_policy.is_exiting_allowed() returns true


It's more like,

- all routers in the past 16 consensuses
- for which stem's exit_policy.is_exiting_allowed() returns true

but it also includes all ips it has ever known for a relay,
https://gitweb.torproject.org/check.git/commit/?id=026e43b08656d78398b15742ddf510f6b17f859e
which seems a little suspect.


> I haven't read any TorDNSEL code and don't know exactly what above
> mentioned TorDNSEL file(s) are but I think they include results from
> active testing of what address exit relays really use for exiting.

Yup, that's it.


> [exit-addresses] seems to be from TorDNSEL too but holding more of a
> snapshot while the files mentioned above supposedly cover some period of
> time.

Yes.


> Before I dig further, perhaps someone here already know why the numbers
> differ so? I wouldn't expect the churn of addresses used for exiting to
> be high enough to explain the difference.

If the numbers don't add up, can you file a bug?