[tor-reports] November 2014 report for the Tor Network Tools team

Sat Dec 6 09:09:08 UTC 2014

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The Tor Network Tools team [1] made the following progress on
services, applications, and libraries related to the status, analysis,
and monitoring of the Tor network:

[1] https://trac.torproject.org/projects/tor/wiki/doc/VegasTeam3

Atlas
- -----

Karsten wrote a small patch to remove advertised bandwidth fraction
graphs from details pages and replace them with middle probability
graphs [2].

 [2] https://trac.torproject.org/projects/tor/ticket/13674

Depictor
- --------

Tom rewrote the old Java code [3] that produces the consensus-health
website [4] in Python [5] and, for now, made the new output available
on his own server [6].

 [3] https://gitweb.torproject.org/doctor.git/log/?h=java
 [4] https://consensus-health.torproject.org/
 [5] https://gitweb.torproject.org/depictor.git/
 [6] http://utternoncesense.com/

DocTor
- ------

Damian extended DocTor to notify directory authority operators
directly of issues in addition to the tor-consensus-health@ list.
This should reduce duration of outages and other issues with the
authorities.

Damian replaced turtles in DocTor with longclaw [7].

 [7] https://trac.torproject.org/projects/tor/ticket/13296

Damian properly fixed DocTor's OOM issues by not shelling out to send
notification emails [8].

 [8] https://gitweb.torproject.org/doctor.git/commitdiff/94a2e9d

Damian looked into a notice from DocTor where it detected a burst of
relays from Google App Engine.  These relays lacked any contact
information so were dropped from the network as a potential Sybil
attack [9].

 [9]
https://lists.torproject.org/pipermail/tor-consensus-health/2014-November/005252.html

ExitMap
- -------

Philipp refactored the GeoIP and exit relay selection code in exitmap.

Globe
- -----

Isis accepted to become the new maintainer of Globe for a couple
months [10].

 [10] https://trac.torproject.org/projects/tor/ticket/11250#comment:7

Karsten wrote a small patch to remove advertised bandwidth fraction
graphs from details pages and replace them with middle probability
graphs [11].

 [11] https://trac.torproject.org/projects/tor/ticket/13674

Metrics
- -------

Karsten changed navigation towards having a single graph or table per
web page, and added new pages for data files.

Karsten rewrote most of the website text and added a glossary of
frequently-used terms, both of which were kindly reviewed and improved
by harmony.

Navigator
- ---------

Oliver has taken up working on William's Navigator, a Tor status
portal aiming to extend and hopefully replace Globe/Atlas and Compass.
 Oliver is currently working [12] on a feature-complete clone of Globe
using Python and the Flask framework.

 [12] https://github.com/baumanno/navigator/tree/develop

Onionoo
- -------

Karsten continued refactoring the huge class NodeDetailsStatusUpdater
to: fix cases when "running" and "hibernating" fields in details
documents were not updated correctly; prepare supporting bulk imports
better; and make the code there more readable and extensible (#12651).
 iwakeh did some reviews of these changes.

Karsten merged and deployed iwakeh's patch to use an embedded web
server/servlet engine [13].

 [13] https://trac.torproject.org/projects/tor/ticket/13089

Karsten rebased and deployed a patch from October to support
base64-encoded fingerprints in the `search` parameter [14].

 [14] https://trac.torproject.org/projects/tor/ticket/13135

Karsten wrote and deployed two patches to remove the advertised
bandwidth fraction field from relay details documents [15] and to
remove advertised bandwidth and advertised bandwidth fraction graphs
from weights documents [16].

 [15] https://trac.torproject.org/projects/tor/ticket/13673
 [16] https://trac.torproject.org/projects/tor/ticket/13674

Stem
- ----

Damian reviewed and merged some delightful improvements made by Ossi
Herrala to parse descriptors ~10% faster [17], and to read from the
'.new' files in Tor's data directory [18].

 [17] https://trac.torproject.org/projects/tor/ticket/13821
 [18] https://trac.torproject.org/projects/tor/ticket/13756

Damian expanded Stem's descriptor tutorials [19, 20] with an example
of persisting descriptors to disk, based on an idea by mmcc [21].

 [19]
https://stem.torproject.org/tutorials/mirror_mirror_on_the_wall.html#saving-and-loading-descriptors
 [20]
https://stem.torproject.org/tutorials/examples/persisting_a_consensus.html
 [21] https://trac.torproject.org/projects/tor/ticket/13774

Damian replaced quite of bit of boilerplate in Stem with a new
@with_default decorator [22].

 [22] https://gitweb.torproject.org/stem.git/commitdiff/56a1f03

Damian made some updates to Stem to reflect changes in Tor's spec this
month, including HS_DESC's new reason attribute, and validating the
new consensus parameters.

Sybil attack detector (yet unnamed)
- -----------------------------------

Philipp wrote a project proposal to get funding for a Sybil detector
for the Tor network.  He submitted it to OTF and hopefully will hear
back soon.