[tor-project] Core Tor sponsor 4 June report

[isabela]

In June we finished the task [1] to update dir-spec.txt to reflect
prop278. We continue to run tests, compare measurements and fixed bugs,
some were found at our test network, and an important one (more bellow)
affecting other tor relays.

Some of the bugs we diagnosed and fixed:
    #22460 Link handshake trouble: certificates and keys can get out of
sync [2]
    #22466 "Crosscert is expired" warnings: RSA->Ed25519 identity
crosscertifice apparently made in 1970? [3]
    #22751 LZMA coder causes crash when the sandbox is enabled [4]
    #22752 Assertion failure in consensus_cache_entry_handle_get on
Windows [5]

As mentioned, we found an important bug that cost us some time to figure
out. This was related to the implementation of both proposals, 140 and
278. Where we didn't send the "not modified" header so relays would keep
asking for data they already had and receive the document. This was
leading to more bandwidth usage which is completely the contrary of our
goal with this project.

The bug was tracked under this ticket:
    #22702 Never send a consensus which the downloader already has [6]

We collected measurements before and after fixing the bug [7]. If you
look at our spreadsheets, in the "regression found" subsheet you will
see that middle nodes (XXXm under ID) and relays (XXXr under ID) are
above 100% in the delta, which is a regression (using more bandwidth).
In the "regression fixed" subsheet, you can see that for XXXm and XXXr
nodes this is fixed. They are around 55% which give us a bandwidth
improvement of 45% for relays.

For clients we ave reduced the bandwidth consumption to the point where
we are now only using 41% of the bandwidth we used before version 0.3.1
of core tor.

We are very happy with these results. For next month we will evaluate
one more compression algorithm to see if we can improve these gains even
more[8]. And we also received a ticket[9] from an user (relay operator)
with a low-powered machine complaining about a side effect our solution
have created. Where a relay now generates a bunch of diffs every time it
gets a new consensus, and then it compresses them all in several ways,
this happens in the background, but for low-powered machines is
noticeable as it's taking a lot of resources. We have a couple of design
changes to solve this in mind and will be working on it in July as well.

[1] https://trac.torproject.org/projects/tor/ticket/22275
[2] https://trac.torproject.org/projects/tor/ticket/22460
[3] https://trac.torproject.org/projects/tor/ticket/22466
[4] https://trac.torproject.org/projects/tor/ticket/22751
[5] https://trac.torproject.org/projects/tor/ticket/22752
[6] https://trac.torproject.org/projects/tor/ticket/22702
[7]
https://docs.google.com/spreadsheets/d/1l_m-kHkOrq4UGmTUtpuffbUY85fvcDQJaoF3k9t-8r8/edit#gid=721942207
[8] https://trac.torproject.org/projects/tor/ticket/22819
[9] https://trac.torproject.org/projects/tor/ticket/22883