[tor-dev] Pyonionoo Descriptors

[Karsten Loesing]

Hi Megan, hi Erik,

cc'ing Sathya and tor-dev, hoping you don't mind.

On 7/27/12 5:21 PM, Megan Chang wrote:
> We are currently almost done with the first draft of pyonionoo.

Awesome! :)

> We were
> wondering what some of the entries might look like so we would be able to
> see what information was being parsed to fill in some of the fields such as
> the bandwidth handler's "write_history" and "read_history". We noticed that
> the router descriptor format includes "write_history" and "read_history"
> according to torspec but again, we aren't too sure on what these entries
> look like.

The bandwidth documents are tricky, because one cannot just look up
these values from Tor descriptors.  They're based on histories in
extra-info descriptors, but they aggregate these values on different
levels of detail, and they're specifically designed for graphing.  I
don't know about your schedule, but if you're short on time, I'd say
let's discuss summary and details documents first and ignore bandwidth
documents for the moment.  (Of course, if you have plenty of time left,
happy to discuss bandwidth documents more!)

Is your code online somewhere?  I'd like to take a look and give you
some feedback about Tor descriptor specific things (not so much about
Python/Twisted (?) things).

> Also, we don't know how to approach the fields in the details
> handler that require GeoIP information. Any tips on this?

Onionoo uses Maxmind's Java API, their GeoIP database, and their AS
database to resolve IP addresses to country/AS information.  There's a
Python API, too, which I think Sathya used for his network entropy code:

https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-6232/pygeoip

If you're unsure how to use it, best ask Sathya for more details.

Best,
Karsten