[tor-dev] Arturo's June Status Report

[Arturo Filastò]

# Summary

I worked mainly on documentation and specification tasks related to 
OONI. Did a bit of travelling and a bit of coding too.

# Documentation/Design related

The wiki is starting to look good. Most of the basic analysis required 
for moving on to implementing
OONI is in place. We still need to have some more discussion on a few 
design choices, but the overall
concept is in place.

In particular we have finished the analysis of Tests.

I created templates for the analysis of Censorship detection tools 
(https://trac.torproject.org/projects/tor/wiki/doc/OONI/CensorshipDetectionTools/Template)

did the analysis of Neubot: 
https://trac.torproject.org/projects/tor/wiki/doc/OONI/CensorshipDetectionTools/NeuBot
and of Herdict: 
https://trac.torproject.org/projects/tor/wiki/doc/OONI/CensorshipDetectionTools/Herdict

We created the methodology for specifying and writing OONI probe tests:
https://trac.torproject.org/projects/tor/wiki/doc/OONI/TestWritingMethodology

created a module view and a component and connector view and specified 
the overall
architecture of OONI: 
https://trac.torproject.org/projects/tor/wiki/doc/OONI/Architecture

I started defining the API for the backend system in particular the 
reporting mechanism:
https://trac.torproject.org/projects/tor/wiki/doc/OONI/Backend.

In the past days had some discussions with jake and isis if we should be 
using HTTP for reporting.
Jake suggested that we use rsync instead of HTTP. I will detail in a 
following mail the pro and cotra
of one solution over the other.

We finished specifying the basic set of Test for OONI here: 
https://trac.torproject.org/projects/tor/wiki/doc/OONI/Tests

The data collection scheme has been defined here: 
https://trac.torproject.org/projects/tor/wiki/doc/OONI/DataCollection.

I got a few good tips from Nick on the topic of design and specification 
of software. What I took from it is that in the end the design that you 
make will not be ultimate and you will end up changing it. For this 
reason it's good to start implementing some of the tests and that will 
allow you to understand what is the abstraction that fits all of them.
He used a very nice analogy, it's like having a set of points in N 
dimensional space and by designing an abstraction for it you are trying 
to find an hypercube that fits all of them (I may have a bit 
re-elaborated this).

# Coding

Implemented an HTTP utility library for writing HTTP based OONI tests 
and implemented a few of the using
such library.

Wrote a OONI scapy twisted library that replaces scapys send and receive 
functions with some based on
twisted. Wrote an OONI Test protocol for it.

Implemented based on OONI a test written by phw that does packet 
mutation to bissect censorship fingerprints.

Misc. code refactoring.

# Travel/Advocacy

Travelled to Madrid to do a talk on Tor and GlobaLeaks. Met there with 
Jillian York.

Went to hackmeeting in l'Aquila and did a talk about Tor and censorship 
( http://www.slideshare.net/hellais/tor-censorship-2012-ooni). I believe 
this talk went very well and everybody was very excited about Tor at the 
end of it.

# What I plan to do next

Work on drawing these N dots, by implementing test for OONI. I will 
finish one of the first fully client server tests based on OONI called 
b0wser (the server part is almost done). Figure out what should be done 
with the backend and start having a test backend deployment on ooni.nu.

- Art.