[tor-scaling] Nov 20 meeting recap: metrics analysis needed

[Karsten Loesing]

Hi!

On 2019-12-02 20:37, Mike Perry wrote:
> In particular, I would like to look at 8 hour snapshots from 8/5 to
> 8/15, broken out by relay flags, of CDF-TTFB and CDF-DL from
> https://trac.torproject.org/projects/tor/wiki/org/roadmaps/CoreTor/PerformanceMetrics

This is a very interesting experiment, and I'd like to help with the
analysis, also in preparation of providing better tools for future
analyses like this one.

However, I'm having trouble understanding what graphs you have in mind
here. I made a very first graph, even though I think it's just the
beginning of an interactive process (that should probably not happen on
this mailing list but on a ticket).

https://people.torproject.org/~karsten/onionperf-cdf-ttfb-2019-12-19.pdf

Some thoughts:

 - I'm not sure if CDFs are the best way to visualize the data here.
CDFs work great for visualizing *one* distribution or for comparing a
handful of distributions when plotting them as separate lines. But we
have a few dozen distributions here, and we can't plot all these lines
into a single coordinate system.

 - Maybe we could try out different subsets of percentiles in a common
line plot to see how the experiment affects TTFB in this case. Something
like five-number summary or seven-number summary or whichever handful of
percentiles we want to see.

 - I didn't understand your idea to break out data by relay flags. These
are requests over three-hop circuits. How would we split these up by
relay flags?

 - This graph shows just the data from a single OnionPerf source. If we
add multiple sources, it gets even more overloaded. But we can't really
mix numbers from different sources, as they have their very own
connection characteristics that would skew results.

Happy to make more graphs. It might help to see a sketch or longer
description of what you expect to see. Thanks!

All the best,
Karsten

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 528 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tor-scaling/attachments/20191219/677d56c6/attachment.sig>