[tbb-dev] Running xpcshell tests

[Georg Koppen]

Nicolas Vigier:
> Hello,
> 
> I am currently investigating how we can run the Mozilla xpcshell tests
> on our version of firefox, and the other unit tests.
> 
> The xpcshell tests are included on the firefox source tree, and require
> a complete build tree, built without the '--disable-tests' option. So
> we cannot run those tests on the version that is shipped in the bundle.
> 
> To run those tests, I can see different options:
> 
> 1. Run the tests as part of the gitian build process and save the output
>    in a file that can be analyzed later.
> 
>    The bundle build process is already very long, and we probably don't
>    want to make it even longer. Running the xpcshell tests on my laptop
>    (i5-2520M CPU @ 2.50GHz) took 2.5 hours. So this is probably not a
>    good idea.
> 
> 
> 2. Save the build tree in a tarball at the end of the build, and make it
>    available with the bundle files. We can then use it to run the tests
>    as part of our test suite.
> 
>    The main problem with this is that the obj directory at the end of
>    the build is taking 6GB. It's possible to reduce it to a 888MB tar.xz
>    file, but it takes 50 minutes to create it on my laptop.
> 
> 
> 3. Build the browser separatly, in the VM that we use for running the
>    tests. We enough test VMs to be able to do that.
> 
>    This also allow us to run those tests on any commit.
> 
>    The main problem with this is that we are not running the tests on
>    the same binaries that are shipped in the bundle.
> 
> 
> So I'm thinking about doing 3. Does anybody have any other idea or
> comment about this ?

Assuming all commits make it into nightlies first I think I am a fan of
having the tests as part of our regular nightly builds. I am even in
favor of having the latter as debug builds by default which would help
us debugging test failures more easily, too. But that is an orthogonal
issue. Anyway, both could easily be achieved by a special .mozconfig
file and some minor patching of our browser descriptors (like "take the
nightly .mozconfig iff we are about to build a Tor Browser for
versions.nightly")

Georg

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tbb-dev/attachments/20140714/a88c0e55/attachment.sig>