[tor-bugs] #30801 [Core Tor/Tor]: Investigate running CI with hardened dependencies vs running CI with valgrind

[Tor Bug Tracker & Wiki]

#30801: Investigate running CI with hardened dependencies vs running CI with
valgrind
------------------------------+--------------------------------
     Reporter:  nickm         |      Owner:  (none)
         Type:  task          |     Status:  new
     Priority:  Medium        |  Milestone:  Tor: 0.4.2.x-final
    Component:  Core Tor/Tor  |    Version:
     Severity:  Normal        |   Keywords:  tor-ci
Actual Points:                |  Parent ID:
       Points:                |   Reviewer:
      Sponsor:                |
------------------------------+--------------------------------
 In #30674, we investigated why running with --enable-fragile-hardening had
 missed a memory leak that valgrind could successfully catch.  The answer
 turned out to be that we had not compiled our dependencies with sanitizers
 enabled -- so they didn't catch memory leaks that happened inside our
 dependencies.

 Assuming we want CI to catch this kind of bug (and we do!) the
 alternatives seem to be: build our dependencies with sanitizers, or run
 with valgrind.

 Teor made the following notes about deployment and evaluations:

 > Hardened dependencies:
 > 1. We know we can harden dependencies
 > 2. Hardened dependencies may cause CI failures due to bugs in
 dependencies
 > 3. Hardened dependencies may be slower
 > 4. We probably won't rebuild libc and other large libraries in hardened
 mode
 > 5. We don't know if valgrind or hardened builds provide better coverage
 of the kinds of coding errors we typically make
 > 6. It might be complicated to configure builds for all our dependencies
 > 7. We can't harden our chutney, stem, and sbws CIs, because they use
 pre-built binaries
 >
 > Valgrind:
 > 1. We don't know if valgrind runs well in Travis CI
 > 2. Valgrind may cause CI failures due to bugs in dependencies
 > 3. Valgrind may be slower
 > 4. Valgrind instruments all the code, no matter which library it's in
 > 5. We don't know if valgrind or hardened builds provide better coverage
 of the kinds of coding errors we typically make
 > 6. Valgrind is simple to configure
 > 7. We can run valgrind on the pre-built binaries in our chutney, stem,
 and sbws CIs

 We should come to a decision here and take action.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/30801>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online