[tor-project] PieroV's Monthly Status Report, June 2024
Pier Angelo Vendrame
pierov at torproject.org
Mon Jul 1 07:33:59 UTC 2024
Hi everyone!
Here is my status report for June 2024.
This month, I worked almost exclusively on the ESR transition.
Once we got the tags for Firefox 128.0b1, I rebased my RR branch once
again and opened the MR [0] to merge it to our first tor-browser-128 branch.
After that, I finalized the document with comments on the range-diff
converted to HTML with the procedure described in my previous report.
It forced me to be more meticulous about every change, and I found some
errors I had initially missed.
You can find it in the linked MR.
After that, I switched to the tor-browser-build part. First for desktop,
then for Android. Linux and macOS were smooth, and the builds were
reproducible at the first attempt.
Windows was more involved for a couple of reasons. First, Firefox now
requires the windows-rs crate. Mozilla decided not to vendor it for some
reason I haven't investigated.
Then, I had problems with llvm-windres. I solved this by restoring the
wrapper the upstream project deleted some years ago [1]. Indeed, Mozilla
is still using it, but their version would break OpenSSL, so I used a
more recent one.
After this, I could get a build, but it wasn't reproducible. I noticed
that only some binaries were different, and my hypothesis was that it
was binaries that included some Rust code.
We verify reproducibility by building in different machines (I used my
workstation and one of our build servers). We don't share any artifacts:
every machine has to compile everything from scratch (we bootstrap
several compilers, including Rustc).
So, I verified my conjecture using the Rustc artifact I created on my
computer on the build server, and it solved the reproducibility problem.
The differences were only in std, so I think they were due to the
differences in the GCC-based mingw-w64 toolchain.
We've had a ticket about building Rustc with the LLVM-based mingw
toolchain we use for everything else [2] for a long time, but it isn't
trivial due to Rust's hard dependency on libgcc. A series of new target
triples [3] was created to avoid hacking around to resolve this problem,
but Firefox doesn't recognize them (and Mozilla doesn't officially
support this toolchain).
I wrote a small patch to solve the problem, and the build became
reproducible. We will have to discuss more about the disadvantages of
this approach.
The Android part was much more involved. Our builds run offline, whereas
Mozilla's don't. So, I had to write several workarounds, including a
patch for a custom Gradle plugin, and I wrote some Groovy for the first
time in my life 😄️.
The monorepo migration was not troublesome at all: the commands we used
to build in firefox-android.git still work without changes. Eventually,
our build scripts were simplified a little bit.
The Android builds aren't reproducible yet. One reason is that only
GeckoView was patched; our Android Components and Fenix patches haven't
been rebased yet, and they include some changes to make the builds
reproducible. Another reason is that the developments on nimbus-fml
between 115 and 128 introduced a regression, and its output became
non-deterministic. I wrote a patch and opened a PR [4]. I expect all the
reproducibility problems to disappear once our patches are rebased.
The merge request with all the build changes [5] has been reviewed and
approved, but I am waiting for the rebase MR to be merged first.
Apart from the ESR transition work, I rebased 13.0 and 13.5 onto Firefox
115.12.0, did some QA on the 13.5 RCs, and investigated the
reproducibility problem we had with the 13.5 builds.
Cheers,
Pier
[0]
https://gitlab.torproject.org/tpo/applications/tor-browser/-/merge_requests/1043
[1]
https://github.com/mstorsjo/llvm-mingw/commit/8915db817ce728833b8628abadedc7fbc64d5d85
[2]
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/29320
[3] https://doc.rust-lang.org/rustc/platform-support/pc-windows-gnullvm.html
[4] https://github.com/mozilla/application-services/pull/6283
[5]
https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/merge_requests/989
More information about the tor-project
mailing list