[metrics-team] TorDNSEL rewrite using Onionoo

[Arlo Breault]

> On Jan 8, 2018, at 11:06 PM, Philipp Winter <phw at torproject.org> wrote:
> 
> On Mon, Jan 08, 2018 at 06:02:02PM +0000, Iain Learmonth wrote:
>> phw - karsten suggested that you would have an idea on whether exitmap
>> would be a good source for exit addresses? I think it would be good to
>> have one canonical source of exit addresses and then other tools that
>> need them can consume Onionoo's output.
> 
> Yeah, as David mentioned, that's easy to do.  We already have a module
> for that:
> <https://github.com/NullHypothesis/exitmap/blob/master/src/modules/checktest.py>
> 
> The variable check_addr at line 68 contains the exit address, as seen by
> check.torproject.org.


Hi,

It would be great if someone maintained a TorDNSEL-like
service, since our current one if suffering from a
dearth of attention, most likely because it's written
in Haskell.

It lacks IPv6 support [0] and has at least one open bug
resulting in false negatives [1].

At the time that I was actively working on check [2],
I started to rewrite it [3] rather than use exitmap
because exitmap lacked support for continuous scanning [4]
and was not very fast (less important because scanning
each exit at every consensus period probably doesn't
scale anyways).

It should be noted that there's a bit more to running
a service that answers "Tor or not" than serving the
lastest result of an exit scan.  A consensus is valid
for a certain period of time (16hrs?) and a relay could
come back online at any point therein.  Also, oddly,
some relays rotate their exit ip frequently and others
that seem to roundrobin a choice of IPs.  Anyways,
the point is that check tended to err on the side of
false positives for a better user experience.  There's
a bug about the discrepancy [5].

Hopefully the above ramblings are useful.  Let me
know if I can be of help,

Arlo



[0] https://trac.torproject.org/projects/tor/ticket/16947
[1] https://trac.torproject.org/projects/tor/ticket/20114#comment:3
[2] https://gitweb.torproject.org/check.git/
[3] https://github.com/arlolra/exitaddr
[4] https://github.com/NullHypothesis/exitmap/issues/7
[5] https://trac.torproject.org/projects/tor/ticket/18968