[tor-bugs] #29693 [Core Tor/Tor]: Decrease probability of stochastic failures in test-slow

[Tor Bug Tracker & Wiki]

#29693: Decrease probability of stochastic failures in test-slow
----------------------------------------+----------------------------------
 Reporter:  teor                        |          Owner:  asn
     Type:  defect                      |         Status:  assigned
 Priority:  High                        |      Milestone:  Tor:
                                        |  0.4.0.x-final
Component:  Core Tor/Tor                |        Version:  Tor:
                                        |  0.4.0.1-alpha
 Severity:  Normal                      |     Resolution:
 Keywords:  tor-ci, tor-test, 040-must  |  Actual Points:
Parent ID:                              |         Points:  0.5
 Reviewer:                              |        Sponsor:  Sponsor2-must
----------------------------------------+----------------------------------

Comment (by teor):

 Here are our worst-case scenarios:
 * a merge forward from 0.4.0 (or earlier) to master:
   * 3 (maint-0.4.0, release-0.4.0, master) * 14 jobs (travis, appveyor)
   * With N_TRIALS=3 and alpha=1e-06, a travis build will fail with 50%
 chance after 1651 builds.
 * pull requests to 0.4.0 and master:
   * 2 (branch, PR) * 2 (0.4.0, master) * 14 jobs (travis, appveyor)
   * With N_TRIALS=3 and alpha=1e-06, a travis build will fail with 50%
 chance after 1238 builds.

 These error rates are acceptable, but only until we split off maint-0.4.1.
 Then we will need to go to 4 trials, or find some other solution.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29693#comment:11>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online