[metrics-bugs] #33076 [Metrics/Analysis]: Graph onionperf and consensus information from Rob's experiments

[Tor Bug Tracker & Wiki]

#33076: Graph onionperf and consensus information from Rob's experiments
-------------------------------------------------+-------------------------
 Reporter:  mikeperry                            |          Owner:
                                                 |  metrics-team
     Type:  task                                 |         Status:
                                                 |  needs_review
 Priority:  Medium                               |      Milestone:
Component:  Metrics/Analysis                     |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  metrics-team-roadmap-2020Q1, sbws-   |  Actual Points:  2
  roadmap                                        |
Parent ID:  #33121                               |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by karsten):

 Dennis brought up the questions whether failed measurements are included
 in the CDF-TTFB graph or not. They are not right now, but we might
 consider including them with `TTFB=Inf`. The result would be that the
 curve doesn't go to 100% when there are failed attempts. That would give
 us an idea of when to expect the first byte in x% of measurement
 ''attempts''. However, a possible downside might be that different time
 periods are harder to compare when there was a higher rate of failures in
 one period. On the other side, maybe that's useful information.

 I [https://trac.torproject.org/projects/tor/attachment/ticket/33076
 /onionperf-cdf-ttfb-2020-02-11.pdf attached] two variants of CDF-TTFB, one
 containing ''all successful'' measurements as before and one containing
 ''all'' measurements including the failed ones. The difference is really
 small in this case, visible for example in the op-us onion graph. It might
 be more visible in other cases.

 Leaving this here for discussion. This might be a non-standard way of
 using ECDFs and therefore harder to understand and possibly harder to make
 with matplotlib. But if there's agreement that it would be good to have,
 we should try to make these graphs.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/33076#comment:21>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online