[tor-bugs] #27218 [Applications/Tor Browser]: Make rebundling faster by generating multiple bundles in parrallel

[Tor Bug Tracker & Wiki]

#27218: Make rebundling faster by generating multiple bundles in parrallel
-------------------------------------------------+-------------------------
 Reporter:  boklm                                |          Owner:  tbb-
                                                 |  team
     Type:  task                                 |         Status:
                                                 |  needs_review
 Priority:  Very High                            |      Milestone:
Component:  Applications/Tor Browser             |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  tbb-rbm, boklm201810,                |  Actual Points:
  TorBrowserTeam201810R                          |
Parent ID:                                       |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------
Changes (by boklm):

 * status:  new => needs_review
 * keywords:  tbb-rbm, boklm201809, TorBrowserTeam201810 => tbb-rbm,
     boklm201810, TorBrowserTeam201810R


Comment:

 There is a patch for review in branch `bug_27218_v3`:
 https://gitweb.torproject.org/user/boklm/tor-browser-
 build.git/commit/?h=bug_27218_v3&id=787cc941f929af286ebbe595adee92051dfd566b

 This branch is based on #28002, so it needs to be merged after #28002, or
 cherry-picked.

 With this patch, instead of directly generating the bundles, we instead
 create one separate script to generate the bundle for each locale, and use
 `run_scripts` to run those scripts in parallel.

 I compared the time to re-bundle an alpha with and without this patch on
 my build machine:

 - linux-x86_64. Without patch: 1:23:46 With patch: 22:25
 - linux-i686. Without patch: 1:27:57 With patch: 23:00
 - windows-x86_64. Without patch: 57:57 With patch: 17:04
 - windows-i686. Without patch: 1:02:18 With patch: 18:41
 - osx-x86_64. Without patch: 34:10 With patch: 11:28

 So the total rebundle time without this patch was 5 hours and 23 minutes,
 and it was 1 hour 31 minutes with the patch.

 I also compared the build output from this branch and current master. The
 result was the same, except for the en-US Windows installer, because it
 included the fix for #28002. I will now do a build with only the patch for
 #28002 to check that I get the same `en-US` bundle.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/27218#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online