[tor-dev] Tor project automation work

[Georg Koppen]

Hi Nicolas,

some remarks are below.

Nicolas Vigier:
> In order to help me doing that, I'm very interested to receive from
> developers of any tor components :
> 
> - a description or ticket number of bugs that you wish could have been
>   detected earlier with automated tests

https://trac.torproject.org/projects/tor/ticket/8143 comes to mind here.

> - any wish or specific needs that you may have

You write: "Tor Browser includes some patches to make the build
reproducible. We could have a test that check the reproducibility of the
build by building the browser twice."

While this is indeed a good idea, it won't be enough as we had bugs in
the past that were only visible when builds on different machines got
compared. So, what I'd like to have (in addition to running browser
builds twice? on different machines?) are tests that cover specific bugs
we avoided (see: the "Remaining Build Reproducibility Issues" in Mike's
blog post covering the technical details of the Gitian build) or tracked
down. See: https://trac.torproject.org/projects/tor/ticket/10159 for an
example for the latter. (There, one could write a test that automates
the creation of the browser.manifest which would eventually (i.e. if run
a couple of times) show whether this bug still exists or whether not).)

> - anything else that you think might be useful for me to know

You write: "We can produce some packages for Tor Browser, to make
testing of the browser easier."

In which regard is it easier to test the browser if you have packages?
And how should that look like outside of the Tor Browser Bundle? I fear
we create extra bugs if we move the browser outside the environment it
will be used in (i.e. the TBB) just for testing purposes. Even if we
won't create extra bugs this way we might miss some. To be clear,
running the tests you call "Usablility tests" and "Reproducible build
test" outside the bundle, seems fine to me, while my concerns apply to
the fingerprinting and privacy tests.

The test helpers are a good idea. You might want to rename "Cookies
tool" to something like "Identifier tool" matching the Tor Browser
specification more closely (especially as you actually mean "Identifier
tool" as "Later versions could be extended to also use other techniques
for storing informations in the browser" indicates).

Georg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20131209/1b73d0fc/attachment.sig>