[tor-dev] Automating Bridge Reachability Testing (#6414)

[Arturo Filastò]

On Fri, Oct 12, 2012 at 10:48 PM, Isis <isis at torproject.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> Hi Karsten!
>
> Oh sheesh. I did not see it...I will have to figure out why. That is slightly
> worrying.
>
>
>     2) Arturo redesigned the OONI testing framework API again to use a
>     completely different structure, which was supposed to be backwards
>     compatible and turned out not to be (though I believe that my recent OONI
>     commits fixed that).
>        However, I have been fighting the framework already, because the main
>     scripts in OONI (/ooni/oonicli.py and /ooni/ooniprobe.py) control the
>     reactor, and also expect static iterations through single test and single
>     control functions for each asset (an asset in this case would be one
>     bridge address).

What do you mean exactly by the fact that your tests control the
reactor? The twisted reactor is an event loop and it should be started
only once and stopped only once. Can you show me the piece of code
that you wrote that "controls it"?
There is a proper way to do more or less anything, but some things
require gotchas.

>     The bridge testing is rather dynamic (I would like it to
>     be able to evaluate an approximate danger level to running the next test)
>     and so the framework is kind of troublesome. Also, because the framework
>     handles calling the reactor (in Twisted, the reactor is a sort of event
>     scheduler), and it also expects a rather linear progression of
>     defer.Deferreds (in Twisted, those are standin objects which execute
>     callbacks when they get results from some previous deferred/callback), it
>     would be nicer if I had full control of these myself without needing to
>     hack around the parent scripts.

The twisted reactor is not like a scheduler. It is an event loop. This
means that you register deferred with it (promises that your function
will eventually return) and when you fire the event for the fact that
your operation has concluded it will run the callbacks.
This pattern is common to (I think) all event loops (like libevent for example).

The neat thing though is that the new API actually supports
"automatically" blocking operations transparently. This means that if
you run a test that is blocking and you don't want to fiddle around
with twisted deferreds you can do it none the less.
Check out the documentation for it https://ooni.readthedocs.org and
for more details on what is possible you can do most of what is
possible with twisted trial:
http://twistedmatrix.com/documents/current/api/twisted.trial.html.

If you give me some examples of where you are encountering
difficulties I can help you out.

>     I think it's wise that OONI deals with
>     these things for the testwriter in most cases, because the testwriter
>     shouldn't be expected to be an expert in using Twisted. However, I also
>     think that, in the long term, OONI shouldn't prohibit people who know what
>     they are doing or are doing odd things from being able to do so.

Being a framework I believe this is not the ideal path. A framework is
made to impose on you certain design decisions that you do not have to
make. There is a right way to use such frameworks and if you don't use
them as they are designed you will obviously have some issues.

Could you provide me with examples of what you are trying to do and
you are not able to do or experiencing problems with?

Whatever you need to do should be possible and if it is not possible
this functionality should be exposed to whoever will use OONI in the
future.

>        As a result, I've decided (for now), to use bits are parts of the OONI
>     code before the recent refactoring, and later (after the deliverable) I
>     will work on adding flags to OONI to give the test script full control of
>     the reactor and deferreds, as well as evaluating whether or not the bridge
>     test is even compatible with the new API. I do not want to get caught up
>     in dealing with this right now, I just want to have it all working and
>     deployable in a way that I know will work.

I still do not understand exactly what you mean by "full control of
the reactor". I am very keen in learning what you are trying to do
though and I bet there is a way to do it, since the new OONI is just
an extension of twisted trial unittesting framework to support inputs
and reporting with the OONI format.

- Art.