[metrics-bugs] #34023 [Metrics/Onionperf]: Reduce the number of 50 KiB downloads

[Tor Bug Tracker & Wiki]

#34023: Reduce the number of 50 KiB downloads
---------------------------------------+--------------------------------
 Reporter:  karsten                    |          Owner:  metrics-team
     Type:  enhancement                |         Status:  needs_review
 Priority:  Medium                     |      Milestone:
Component:  Metrics/Onionperf          |        Version:
 Severity:  Normal                     |     Resolution:
 Keywords:  metrics-team-roadmap-2020  |  Actual Points:
Parent ID:                             |         Points:  1
 Reviewer:                             |        Sponsor:  Sponsor59-must
---------------------------------------+--------------------------------

Comment (by karsten):

 With #26673 being resolved we now have partial completion times available.
 That means that we have timestamps when a 5 MiB download completed the
 first 50 KiB or 1 MiB and when a 1 MiB download completed the first 50
 KiB.

 I reprocessed past measurement results from op-hk2, op-nl2, and op-us2
 from the first two weeks of May 2020 to see whether we can use partial
 completion times of larger downloads together with full download
 completion times.

 Here are some remarks on the
 [https://trac.torproject.org/projects/tor/attachment/ticket/34023
 /onionperf-partials-2020-05-16.pdf attached PDF]:

  1. The first plot shows number of measurements without (green) and with
 (purple) including partial downloads. The number of 50 KiB measurements
 increases by about 1/5, which includes both 1 MiB and 5 MiB downloads. The
 number of 1 MiB measurements increases by about 1/2 as compared to before,
 which only includes the 5 MiB downloads. The number of 5 MiB downloads
 stays the same, because there are no larger downloads than 5 MiB.

  2. The second plot shows ECDFs of time to download 50 KiB. Each of the
 subplots contains two lines, one in green and one in purple. They are just
 so similar that they're basically indistinguishable. The purple line
 contains 20% more data points than the green line, but that doesn't make
 any visible difference.

  3. The third plot shows ECDFs of time to download 1 MiB. Interestingly,
 there are some minor differences between the green and purple line. The
 reason is that the purple line contains 50% more data points than the
 green line. That's a larger difference than the additional 20% in the 50
 KiB case. Still, there doesn't seem to be systematically different
 measurements when including partial completion times or not.

  4. The fourth plot shows ECDFs of time to download 5 MiB. I only included
 this as a way to sanity check the plotting code. Green and purple lines
 are exactly the same here, because we have as many partial completion
 times for 5 MiB downloads as full download completion times.

 Let's discuss how to proceed with these results. My recommendation is that
 we:

  1. extend the [https://metrics.torproject.org/torperf.html "Time to
 download files over Tor" graph on the Tor Metrics website] to include
 partial completion times, if available, and

  2. modify deployed OnionPerf instances to ''only'' download 5 MiB files
 and no 50 KiB or 1 MiB files anymore.

 Leaving in needs_review to get feedback. I'll also bring this ticket up at
 the next weekly meeting on Thursday.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/34023#comment:9>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online