[tor-dev] Local onionoo cache

[Karsten Loesing]

On 5/12/13 12:38 PM, David Fifield wrote:
> Karsten, could I ask you to take a quick look at this code?
> 
> https://trac.torproject.org/projects/tor/ticket/7549#comment:14
> https://trac.torproject.org/projects/tor/attachment/ticket/7549/onionoo-querying-ver2.zip
> 
> It's a daemon that keeps a local cache of potential exit relays, sourced
> from onionoo. It listens on a local port and sends "EXIT" when asked
> about an IP address that is possibly an exit. The idea behind this code
> was to have a fast local database that we can rapidly query from the
> flash proxy facilitator, in order to prevent Tor users from being flash
> proxies themselves.
> 
> Is this code the most straightforward way you can think to achieve the
> goal?

I think so, yes.

The only downside I can see is that it takes about 30--45 minutes for
new exits to show up in your local cache.  An alternative would be to
query the exit list yourself, download the most recent consensus, and
compile a list of exit addresses yourself.  But that's probably too
complicated for the purpose.  (A downside of that approach, however, is
that you'll have to change your code once TorBEL will be deployed.)

> Do you have any suggestions on the use of onionoo?

The code looks sane to me.  The only improvement might be to lower
ELAPSED_UPDATE_TOR_NODES_TIME to, say, 300 or 600 seconds.  Onionoo
updates its data once per hour, and with the current 3600 seconds you
might be unlucky and download its data right before it gets updated.
Given that you're sending the If-Modified-Since header, querying every 5
or 10 minutes (or even more often) is perfectly fine.

> I wonder if
> something like the daemon exists already.

I'm not sure, but Tor2web might do something similar.  From Onionoo's
project page: "Tor2web is a web proxy to Tor Hidden Services. It uses
Onionoo to get the list of currently running Tor Exits to detect if the
client is a Tor user and if so redirect them to the .onion address."

> The code looks reasonable,
> though I would make some changes before merging it. I want to gauge
> whether detection of exits is worth the additional code.

Hope this helps you decide.  If you plan to use Onionoo, please let me
know, so that I can put flash proxy on the list of Onionoo clients and
remember to inform you of upcoming protocol changes.

Best,
Karsten