[tor-bugs] #4439 [Metrics Utilities]: Develop a Java/Python API that wraps relay descriptor sources and provides unified access to them

[Tor Bug Tracker & Wiki]

#4439: Develop a Java/Python API that wraps relay descriptor sources and provides
unified access to them
-------------------------------+--------------------------------------------
 Reporter:  karsten            |          Owner:  karsten
     Type:  task               |         Status:  new    
 Priority:  normal             |      Milestone:         
Component:  Metrics Utilities  |        Version:         
 Keywords:                     |         Parent:         
   Points:                     |   Actualpoints:         
-------------------------------+--------------------------------------------

Comment(by karsten):

 Replying to [comment:1 atagar]:
 > I would be interested in discussing this with you during the winter
 developer meeting.

 Happy to talk to you earlier than that.  I really hope we're done with the
 first version of this API long before the dev meeting, or I'll have to
 copy the same code a couple more times.

 > One thing that I'd like (and maybe it's tangential to this) is a library
 I can run locally that provides a facade with equivalents to the GETINFO
 ns/*, desc/*, and microdescriptor options (the last pending ticket 3832).

 Not tangential.  I think that's pretty much what the API should do.

 > This library could then read from the control port, cached-* files, or
 the metrics service.

 cached-* files are fine, as well as directory authorities/mirrors that I
 listed in the ticket description.

 What's the advantage of talking to the control port if we can read the
 cached-* files?  Shouldn't be hard to add this as a third data source,
 though.

 By metrics service you mean the metrics database?  This will have to wait
 until we have a good relay descriptor database.  See #2922 and #4440 for
 my thoughts on that.  I agree that adding a database as the fourth data
 source would be a neat extension of the API.

 > At present we tell scripts that just need consensus information to run a
 tor client with 'FetchDirInfoEarly' which does the trick, but it seems
 like we can come up with a better and lighter weight alternative to
 subscribe for getting the latest consensus.

 Yes.  That would be downloading from the directory authorities/mirrors.  A
 tricky part will be to explain to the users of this API which data source
 to use when.

 > This would be a start of the metrics service API, with obvious next
 steps being to fetch historical consensus data or aggregated statistics.

 Historical consensus data are fine.

 But aggregated statistics?  Hmmmmmm.  That means we'll have to specify
 what aggregated statistics there are in the database somewhere.  I like
 the idea.  It's something we'll have to postpone a bit though.

 So, great!  That's really some good input on the idea of writing such an
 API.  Let's write it. :)

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/4439#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online