[tor-dev] Automating Bridge Reachability Testing (#6414)

[Isis]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi Karsten!

Oh sheesh. I did not see it...I will have to figure out why. That is slightly
worrying.

So, I am rushing to meet the final deadline, but I still think it is doable. I
have mostly finished up my OONI work for the month, and I planned to spend the
remainder of this month working on the bridge test.

I have finished most of the actual Tor connection code, as well as one of the
four basic packet level scans (the icmp8 one). Two of the other packet level
scans, the TCP SYN and ACK ones, are pretty much copies of the icmp8 one with
a couple lines changed, and they shouldn't be a big deal. 

There is still the vanilla TLS handshake test/scan/thing, which has not been
started yet, and will take a bit more time than the others because Python
notoriously has problems with SSL bindings and libraries, so I'll need to do a
bit of research on newer ones and updates and see which is the best to use
now. I hear that tlslite[1] is the current best choice; if anyone else has any
input on this it would be very helpful. :)

There were a couple minor hups: 

    1) When George asked me to test pluggable transports, this required
     significantly more refactoring than I previously thought was necessary.

    2) Arturo redesigned the OONI testing framework API again to use a
    completely different structure, which was supposed to be backwards
    compatible and turned out not to be (though I believe that my recent OONI
    commits fixed that). 
       However, I have been fighting the framework already, because the main
    scripts in OONI (/ooni/oonicli.py and /ooni/ooniprobe.py) control the
    reactor, and also expect static iterations through single test and single
    control functions for each asset (an asset in this case would be one
    bridge address). The bridge testing is rather dynamic (I would like it to
    be able to evaluate an approximate danger level to running the next test)
    and so the framework is kind of troublesome. Also, because the framework
    handles calling the reactor (in Twisted, the reactor is a sort of event
    scheduler), and it also expects a rather linear progression of
    defer.Deferreds (in Twisted, those are standin objects which execute
    callbacks when they get results from some previous deferred/callback), it
    would be nicer if I had full control of these myself without needing to
    hack around the parent scripts. I think it's wise that OONI deals with
    these things for the testwriter in most cases, because the testwriter
    shouldn't be expected to be an expert in using Twisted. However, I also
    think that, in the long term, OONI shouldn't prohibit people who know what
    they are doing or are doing odd things from being able to do so.
       As a result, I've decided (for now), to use bits are parts of the OONI
    code before the recent refactoring, and later (after the deliverable) I
    will work on adding flags to OONI to give the test script full control of
    the reactor and deferreds, as well as evaluating whether or not the bridge
    test is even compatible with the new API. I do not want to get caught up
    in dealing with this right now, I just want to have it all working and
    deployable in a way that I know will work.

    3) The indirect scans are becoming quite complicated to automate in any
    sane fashion. I still would like to continue working on this, as I'm quite
    enjoying the difficulty, but due to their temporary and volatile nature
    (they will change frequently depending on the blocking methods of a
    particular country and the currently available in-country
    bounces/proxies/whatever-thing-the-indirect-scan-uses), as well as the
    fact that many of these methods are still undiscovered, I think it is safe
    to add them as specialty cases after the fact without impacting overall
    general testing. There is one in particular that I would like to finish
    before the deadline because I am quite proud of it and am having a lot of
    fun working on it, but I'm first going to concentrate on wrapping up the
    active scans.

There are other things which I've marked as helpful things to do, but which
are not necessarily part of this deliverable:

    1) Having a parser for bridge descriptors to turn them into test inputs,
    and vice versa.

    2) Having some undiscoverable method for setting up lots of IPv6 bridges
    on one OR (Tor currently only allows up to eight, I believe) and having
    these be discoverable by bridgedb and no one else. I was thinking of this
    while talking with Aaron, because he reminded me that people on IPv6 have
    tons of IPs available, and I was thinking that if we configured some type
    of one-way hash function, we could say that a bridge descriptor for
    2001:db8::1:1 should actually mean multiple descriptors for
    2001:db8::fa98:38d2 2001:db8::e099:2188 2001:db8::88aa:3b7 or something,
    derived from the output of hashing the original descriptor with the OR's
    key or something else. This would help distribute bridges in the future
    quite a bit, though it doesn't do much for the current bridge situation.

Anyone wanting to help with the above two things, or with an idea for another
indirect scan, or with feedback on anything I'm working on, should feel free
to contact me and it will be greatly appreciated. :D

.-- .. ... ....
- -- . 
.-.. ..- -.-. -.-
<(A)3
isis agora lovecruft
.-.-.

[1] https://github.com/trevp/tlslite

On Fri 12 Oct 2012 at 14:56, thus spake Karsten Loesing:
> Hi Isis,
> 
> did you see this email?
> 
> Sorry for being pushy, but we're less than three weeks away from the
> deadline, which makes me kinda nervous.
> 
> Thanks,
> Karsten
> 
> 
> On 10/8/12 9:14 AM, Karsten Loesing wrote:
> > Hi Isis,
> > 
> > can you give me a brief status update on the sponsor F year 2
> > deliverable "Automating Bridge Reachability Testing", ticket #6414?
> > 
> > In particular, how did the two project-internal milestones on September
> > 15 and October 7 go?
> > 
> > Do you mind updating #6414 with the status?
> > 
> > The overall deadline is on November 1, which is in 3.5 weeks.  How
> > realistic do you think it is that we'll have automated bridge
> > reachability tests by then?
> > 
> > Thanks,
> > Karsten
> > 
> 
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iQIcBAEBCgAGBQJQeJ4pAAoJEKOttnos24s1GlkP/1ODucFe9q9a7KaKisgVqBcx
Cg1X3yGI+dsWvKtMwzgNqT/hcS+95GAYQiaa8fDlbiLY3nCWEsGYSOTjsbAmp04O
qPCdBiSFE4VqiZy4PYUHUkf8qZTYkXqAiMnyuuLVJINJ+BHLAdwEmEWB77ncQUxV
PvG6sxQ3GY64ZqRaSHwudf3H8mEPN/tNWRIXubnvAGM16EV0hWb51uORwo6CG86m
lKwpBqbSX5uDgG+B6HCeZps0Zv7cDypI3dtdd8hJHncUpFmfuj2y+Oad1M5DVbMC
4UMvCv/4tMPoPyjTYjWgLQRHEFiMdMwQcdPntnF6XqUBMLvd6v1fk1IxaaqNCDah
JWxK8ymUu90FLmmiyaSHQE/tfkFoEDCXz3XzR8xQA1zI2aq0hHdxA+G2JmW8yadQ
wt0N4EeCzK154r4+YRC1eJLrsSOhmA7SaAE6A+/iEOgasdMY0YWchTDnlUbUB5iv
acbMVITSGqmcRMos5xi450V98Qw1nW3UoBj5wi8uh0BWqnhqfomqL6oloQEdgAze
MqtCZ91adKKXntIsEZW0C/klBYg5mZIxdtmAW5eO/j+ogLB/L0/hX4/TZ6OH5i/N
7jJWAF0MbSWI2Q1Gk+oFFulib5m3XWzqg5Yq/+8VfE5n1R5xM1FgBwC932SWLOFZ
BV5c9ntZ+iJlbhpdL5yJ
=jk13
-----END PGP SIGNATURE-----