[tor-dev] Karsten's status report August 1--31

[Karsten Loesing]

Hi everyone,

August was a long month for me, and I feel like I spent more time
reviewing other people's code than writing my own.  That's really cool!
 We have a bunch of new metrics-related code now, and I hope there will
be even more code contributions for me to review next month!

So, below is what I did in August.  Compared to last month's report, I
left out how much of my developer time I spent on these tasks and added
more links instead.

Best,
Karsten


Added three new graphs to the metrics website: the first two graphs show
our progress in operating 125 fast exits [1] (first half of #6498),
which is a deliverable for sponsor J [2]; the third graph shows
advertised bandwidth and bandwidth history by relay flags [3] (#6671).
Also fixed problems with cutting off enough days from graphs for which
we don't have enough data yet (#6593) and removed GetTor graphs and
related code to reduce future maintenance effort (#6395).

The gentle reader may have noticed that there was no mention of code
reviewing in the paragraph above, but just own code writing.  That's
correct, the metrics website is still a one-man show.  I'm planning to
split up the metrics website beast into a component that produces
JSON-formatted graph data [4] and ask other people to write components
visualizing the data, like Walter Kim's metrics website prototype [5].
Hopefully there will be more code contributions if people don't have to
mess with Java servlets in the current metrics website code.

Reviewed Sathya's great contributions to what we're now calling Compass
[6].  Compass is a website and command-line tool to filter the set of
currently running relays and group by same country or same AS.
It was originally written by delber to analyze Tor network diversity.
Sathya improved Compass by adding options to show only fast exits and
almost fast exits for the sponsor J project to add 125 fast exits [2],
and he wrote the website part.  Since Compass went live, we had a lot of
tickets opened by cypherpunks that we're almost unable to handle [7].
Patches welcome, Sathya and I will review them!

Improved graphs in Atlas [8] to show the displayed quantity on the y
axis (#5388) and added graphs on various probabilities to be selected by
clients (#5455).  Arturo reviewed and cleaned up these patches.
Speaking of code contributions, there are quite a few open tickets [9]
that would benefit from JavaScript developers wanting to help out!

Reviewed Pyonionoo [10], the Wesleyan students' Python rewrite of
Onionoo, the data provider behind Compass and Atlas.  More precisely,
they were working on the front-end portion of Onionoo which gives out
JSON-formatted data, not on the number-crunching back-end that Sathya is
about to rewrite in Python (#6452).  Basic functionality of Pyonionoo
works, but there are a few defects (#6707, #6708) and one enhancement
(#6709) that we'll have to work on before deploying it.  Of course, we
have more open tickets for the Java and/or Python Onionoo [11].

Converted even more old tech reports into the new Tor Tech Report format
[12] (#5405).  The goal here is to have a single place for all reports
we produce and that other researchers can use to cite our work.

Set up Rob's Tor simulator Shadow [13] to see if there are any usability
issues that would prevent researchers from using it.  Turns out it's not
just a very powerful tool to evaluate changes to the Tor code, but also
doesn't require its users to be rocket scientists.  I'm planning to play
more with Shadow in September and walk through the instructions in
Shadow's wiki [14] to find any remaining pitfalls.  And then I'll
encourage Roger and Rob to blog about Shadow.

[1] https://metrics.torproject.org/fast-exits.html
[2] https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorJ
[3] https://metrics.torproject.org/network.html#bandwidth-flags
[4] https://metrics.torproject.org/graphs/relays-by-country
[5] http://tigerpa.ws/tor_metrics/
[6] https://compass.torproject.org/
[7]
https://trac.torproject.org/projects/tor/query?component=Compass&status=!closed
[8]
https://atlas.torproject.org/#details/D223399907113A1F216AAA64997BC1D4CFA8E1AC
[9]
https://trac.torproject.org/projects/tor/query?component=Atlas&status=!closed
[10] https://gitweb.torproject.org/pyonionoo.git
[11]
https://trac.torproject.org/projects/tor/query?component=Onionoo&status=!closed
[12] https://research.torproject.org/techreports.html
[13] http://shadow.cs.umn.edu/
[14] https://github.com/shadow/shadow/wiki