[metrics-bugs] #26673 [Metrics/Onionperf]: Record download times of smaller file sizes from partial completion times

[Tor Bug Tracker & Wiki]

#26673: Record download times of smaller file sizes from partial completion times
---------------------------------------+--------------------------------
 Reporter:  karsten                    |          Owner:  metrics-team
     Type:  enhancement                |         Status:  reopened
 Priority:  Medium                     |      Milestone:
Component:  Metrics/Onionperf          |        Version:
 Severity:  Normal                     |     Resolution:
 Keywords:  metrics-team-roadmap-2020  |  Actual Points:
Parent ID:                             |         Points:  3
 Reviewer:                             |        Sponsor:  Sponsor59-must
---------------------------------------+--------------------------------

Comment (by karsten):

 I started looking into this ticket. I need to take a break now, but before
 I do, here are my thoughts:

  - We're about to stop generating .tpf files in OnionPerf. However, we're
 still using that format in metrics-lib. This means that we can very likely
 ignore generating these new timestamps for .tpf files in OnionPerf, but we
 should still think about doing it in metrics-lib.
  - We should try to generalize from our historic choice of downloading 50
 KiB, 1 MiB, and 5 MiB files by including some more timestamps in the
 middle. There exists something called 1-2-5 series, which could guide us
 here: 10 KiB, 20 KiB, 50 KiB, 100 KiB, 200 KiB, 1 MiB, 2 MiB, 5 MiB.
  - Looking at the .json format, maybe we can have another dict like
 `"payload_progress"` for progress by downloaded bytes: `"payload_bytes": {
 "10240": 0.689492, "20480": 0.949185, ..., "5242880": 2.141004 }"`. Note
 the 6 digits for microsecond precision, unlike the ~16 digits in
 `"payload_progress"`.
  - Maybe we can round `"payload_progress"` numbers to 6 digits for
 microsecond precision as part of this ticket, in a separate commit? It's a
 tiny change, but it makes .json files unnecessarily large.
  - Once this dict is in .json files, I'll have to update metrics-lib to
 parse it. That's going to be a tiny change, though.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/26673#comment:14>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online