[tor-dev] [tor-assistants] Python metrics-lib

[Karsten Loesing]

[Moving this thread to tor-dev as suggested by Damian in separate mail.]

Hi Damian, hi Beck,

On 5/4/12 6:59 PM, Damian Johnson wrote:
> Hi Beck.
> 
>> I noticed that stem has not been updated during the last ten days
> 
> Yup, I've been busy for a while with a new arm release [1] and a
> variety of other chores. Ravi is currently working on safe cookie
> support and I'll be getting back into stem development this weekend.
> 
>> it's a good time to start doing some substantial work
> 
> Great!
> 
>> So for me, a reasonable next step is to build some subclass on top of the base Descriptor class. What do you think?
> 
> Sounds good. What descriptor type would you like to tackle first?
> Karsten, do you have any suggestions? Ravi plans to do network status
> entries and microdescriptors so I'd suggest something besides those.

There are quite a few options.

How about extending stem to implement #2768?  There's already Java code
for verifying server descriptors and consensuses in metrics-tasks.git
[0].  On stem's side this requires parsing the crypto parts of server
descriptors, consensuses, and certificates.  If the network statuses
aren't there yet, starting with server descriptors should be fine.  And
once there's verification code for server descriptors and consensuses,
there are more things to verify in Tor's various descriptors.

Another option would be to write the parsing code for extra-info
descriptors including all the stats.  Once we have consensuses, server
descriptors, and extra-info descriptors in stem, we can start writing a
Python version of Onionoo [1], the service that provides the data for
Atlas [2].

Let me know if you need more options or more input on these.

Thanks,
Karsten

[0] https://gitweb.torproject.org/metrics-tasks.git/tree/HEAD:/task-2768

[1] https://www.torproject.org/projects/onionoo.html.en

[2] https://atlas.torproject.org/


> Also, since you're interested in this area I'd suggest giving this a read...
> https://trac.torproject.org/projects/tor/ticket/5651
> 
> There's nothing that needs to be done for it on our end at the moment
> - just related to the stem/metrics-lib work.
> 
> Cheers! -Damian
> 
> [1] https://blog.torproject.org/blog/arm-release-145
> 
> On Fri, May 4, 2012 at 9:32 AM, Beck Chen <csybeck at gmail.com> wrote:
>> Hi Damian,
>>
>> Greetings! I've read through the metrics-lib and stem.descriptor code and
>> skimmed the directory server specs since you sent me the discussion thread
>> between you and Karsten. I noticed that stem has not been updated during the
>> last ten days, and since I've finished my semester and gone back to China,
>> it's a good time to start doing some substantial work.
>>
>> In stem.descriptor, we already have counterparts for Descriptor.java,
>> DescriptorReader.java, and ServerDescriptor.java now, and as you mentioned,
>> they pretty much behave in the same way. So for me, a reasonable next step
>> is to build some subclass on top of the base Descriptor class. What do you
>> think? I believe you have discussed this with Karsten, so it will be helpful
>> if you can forward me some of your discussions.
>>
>> Beck