[tor-bugs] #25483 [Obfuscation/Snowflake]: Windows reproducible build of snowflake

[Tor Bug Tracker & Wiki]

#25483: Windows reproducible build of snowflake
-----------------------------------+------------------------------
 Reporter:  arlolra                |          Owner:  (none)
     Type:  project                |         Status:  needs_review
 Priority:  Medium                 |      Milestone:
Component:  Obfuscation/Snowflake  |        Version:
 Severity:  Normal                 |     Resolution:
 Keywords:                         |  Actual Points:
Parent ID:  #19001                 |         Points:
 Reviewer:                         |        Sponsor:
-----------------------------------+------------------------------
Changes (by dcf):

 * status:  new => needs_review


Comment:

 I've made progress with cross-compiling webrtc for windows-x86_64. It's in
 two branches. The first branch just upgrades webrtc to branch-heads/64,
 which has the cross-compiling support. The second branch builds off the
 first and adds a build script for windows.

 The build is currently using the prebuilt clang cross-compiler that Google
 puts among the dependencies of webrtc. Eventually we will have to make it
 use our own compiler instead, as we already do on mac and linux. Reliance
 on the prebuilt compiler is the reason why the build currently only works
 for windows-x86_64, not windows-i686: there's no 32-bit prebuilt compiler
 available.

  https://github.com/uumaro/tor-browser-build/commits/webrtc-64 (currently
 [https://github.com/uumaro/tor-browser-
 build/commits/c8048d746ca176808225a428b8820996d65139f8 c8048d746c])::
    This branch updates the build scripts to use a webrtc based on branch-
 heads/64. It depends on having a go-webrtc that also supports the branch-
 heads/64 API. That code hasn't been merged into go-webrtc yet, but there
 is [https://github.com/keroserene/go-webrtc/pull/79 a pull request] for
 it. To test the code before the go-webrtc changes have been merged, do
 this in the tor-browser-build directory before `make`:
    {{{
    cd git_clones/go-webrtc/
    git fetch https://github.com/uumaro/go-webrtc/ branch-heads-64
    cd ..
    }}}
    This branch can be merged independently of the windows code, once the
 go-webrtc pull request goes through.
  https://github.com/uumaro/tor-browser-build/commits/webrtc-win (currently
 [https://github.com/uumaro/tor-browser-
 build/commits/08613622f61c2ece16db1115b14df481c80838a0 08613622f6])::
    This branch lets the webrtc build work for windows-i686. To test it,
 first `get fetch` an updated go-webrtc as described above. If have done a
 build in this directory previously, run
    {{{
    rm gclient/webrtc/.gclient
    }}}
    The .gclient will be regenerated with the addition of `target_os =
 ['win']`, which will cause `gclient sync` to pull the prebuilt clang
 cross-compiler. If you have not yet done a build in this directory, then
 you will need to do
    {{{
    make submodule-update
    }}}
    Then run
    {{{
    rbm/rbm build webrtc --target testbuild --target torbrowser-windows-
 x86_64
    }}}
    If you want to test that other targets still work, do
    {{{
    make testbuild
    }}}

 The output of the build is a file, out/webrtc/webrtc-
 88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz. You
 can compare it against mine:
   https://people.torproject.org/~dcf/pt-bundle/snowflake/webrtc-
 88f5d9180eae78a6162cccd78850ff416eb82483-windows-x86_64-ba8356.tar.gz

 A few comments on the second branch:
  * [https://github.com/uumaro/tor-browser-
 build/commit/7a26f7ae9bde88101f8922c1b340c579a5fd55e6 I compiled my own
 Windows SDK] using the instructions in comment:13 and stashed it in my
 people.torproject.org space. I have [https://github.com/uumaro/tor-
 browser-build/blob/08613622f61c2ece16db1115b14df481c80838a0/TODO.WEBRTC a
 todo] to check if we really need this separate SDK, or if the header files
 in mingw-w64 are enough.
  * The upstream cross-compiling instructions say to put the Windows SDK in
 a case-insensitive [http://www.brain-dump.org/projects/ciopfs/ ciopfs]
 mount, so that, for example, code can `#include <winsock2.h>` when the
 file is actually called WinSock2.h. I tried that initially, but I couldn't
 get ciopfs to work inside the container. boklm gave me a hint that I
 haven't tried yet. In the meantime, though, I [https://github.com/uumaro
 /tor-browser-build/commit/d0f6fcbe6cfec380b8cd3612de52e3a5b6dff8e0
 disabled ciopfs] and worked around case sensitivity with a lot of
 symlinks. I left it in the branch history for now.

 I haven't yet tried actually doing anything with the generated libwebrtc-
 windows-amd64-magic.lib.

 I'm not asking for any of this code to be merged right now. But I would
 appreciate some review of the code (this is my first time writing for
 rbm), and a check to see if the commands above work for you.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25483#comment:14>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online