[metrics-bugs] #34023 [Metrics/Onionperf]: Kill the 50 KiB downloads

[Tor Bug Tracker & Wiki]

#34023: Kill the 50 KiB downloads
-------------------------------+------------------------------
 Reporter:  karsten            |          Owner:  metrics-team
     Type:  enhancement        |         Status:  needs_review
 Priority:  Medium             |      Milestone:
Component:  Metrics/Onionperf  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:                     |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+------------------------------

Comment (by robgjansen):

 **If we remove 50KiB, also remove 1Mib?**

 My main concern isn't the added load on the network, but rather that you
 are removing a metric that we have consistently used as a benchmark for
 the last ~decade. It's useful to be able to compare against a consistent
 benchmark over time.

 I notice from [https://trac.torproject.org/projects/tor/ticket/33076
 #33076] the suggestion that we could use the 1MiB and/or 5MiB results to
 compute the 50KiB times (using the incremental DATAPERC timestamps). That
 seems reasonable to me. Following that logic though, why not remove the
 1MiB file too? I think both the 50KiB and the 1MiB times could be computed
 from the 5MiB results, since we have incremental timestamps for every 10%
 of the download.

 **Reasons to keep all of the files.**

 I think there are two strong reasons to keep all 3 file sizes:
 1. You can specify a different timeout for each of the 3 sizes. That let's
 you cancel the smaller files much sooner if they are hanging. And if the
 timeouts are set realistically, it helps you get a better sense of how
 often we fail to meet a target completion time.
 1. Diversity of circuits. If you follow the suggestion above and remove
 50KiB and 1MiB and only keep 5MiB, and then you get a crappy circuit, data
 points for all 3 download times will be affected. Previously that only
 affected one data point.

 **Adjust download weights instead?**

 If you would like more data points for 1MiB and 5MiB files and fewer for
 50KiB, have you considered adjusting the weights that are used in the TGen
 model file instead of completely removing a file size?
 [https://gitweb.torproject.org/onionperf.git/tree/onionperf/model.py#n90
 The weights are specified here.] For example, if you want all file sizes
 to have equal download probabilities, set the weight for each file size to
 `1.0`.)

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/34023#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online