[tor-bugs] #33076 [Metrics/Analysis]: Graph onionperf and consensus information from Rob's experiments

[Tor Bug Tracker & Wiki]

#33076: Graph onionperf and consensus information from Rob's experiments
-----------------------------------------+--------------------------------
 Reporter:  mikeperry                    |          Owner:  metrics-team
     Type:  task                         |         Status:  needs_revision
 Priority:  Medium                       |      Milestone:
Component:  Metrics/Analysis             |        Version:
 Severity:  Normal                       |     Resolution:
 Keywords:  metrics-team-roadmap-2020Q1  |  Actual Points:
Parent ID:                               |         Points:
 Reviewer:                               |        Sponsor:
-----------------------------------------+--------------------------------

Comment (by karsten):

 I'll have to be AFK for the next couple hours, but until then here's a
 tiny subset of the data I used for the CDF-Relay-* graphs:

 {{{
 fingerprint,validafter,hasexitflag,hasguardflag,read,written,rate,burst,observed,measured
 000A10D4,2019-08-13T09:00,f,f,451,286,102400,204800,112448,2
 000C1F7C,2019-08-13T09:00,f,t,3578899,3591896,19660800,1073741824,8023040,27200
 000CFDEC,2019-08-13T09:00,f,f,1057727,1057703,4194304,6291456,4117504,4360
 0011BD24,2019-08-13T09:00,t,t,17419119,17556018,1073741824,1073741824,34104843,44200
 001524DD,2019-08-13T09:00,f,t,2065558,2119272,5242880,10485760,5455106,9870
 0020D8A2,2019-08-13T09:00,t,t,3686378,3740564,1073741824,1073741824,9030923,11000
 00342C0E,2019-08-13T09:00,f,t,1015702,1020588,4194304,5242880,3717941,4500
 003BFA1B,2019-08-13T09:00,f,t,6623297,6664212,26214400,1073741824,18836759,29800
 003D7882,2019-08-13T09:00,t,f,1280946,1304984,1073741824,1073741824,5083154,3640
 0041E015,2019-08-13T09:00,f,f,175940,176294,10485760,10485760,1102303,994
 00451DF7,2019-08-13T09:00,f,f,539348,539309,4194304,6291456,2820096,3100
 004573FE,2019-08-13T09:00,f,f,15712,15637,819200,1024000,480169,243
 0059D929,2019-08-13T09:00,f,f,82583,82628,409600,1048576,450570,309
 005ED972,2019-08-13T09:00,f,f,8810311,8828818,1073741824,1073741824,22108491,41000
 00723AF1,2019-08-13T09:00,f,t,123404,124200,1073741824,1073741824,21775327,988
 00727F3A,2019-08-13T09:00,f,f,699,698,512000,1024000,240510,2
 0077BCBA,2019-08-13T09:00,t,t,21706519,22054066,1073741824,1073741824,38740943,63000
 0077DDDD,2019-08-13T09:00,f,f,1049,892,13107200,14417920,1969480,3
 008E7B70,2019-08-13T09:00,f,t,3060672,3082137,1073741824,1073741824,9123191,9860
 0095792C,2019-08-13T09:00,f,f,324405,324452,1048576,1228800,963827,1250
 00962D2D,2019-08-13T09:00,f,f,427773,427954,1048576,2097152,1150132,1200
 009851DF,2019-08-13T09:00,f,f,553,391,131072,524288,172355,40
 00B3C9FB,2019-08-13T09:00,f,f,1673,1096,102400,204800,119949,18
 00CCE6A8,2019-08-13T09:00,t,f,355872,375660,1073741824,1073741824,11663560,35
 00D2269D,2019-08-13T09:00,f,t,4218721,4238104,19660800,26214400,9658368,14400
 00E1649E,2019-08-13T09:00,f,t,8092131,8137499,13107200,13107200,12435058,22100
 00E89DDE,2019-08-13T09:00,f,t,1847757,1854884,1073741824,1073741824,6900558,10200
 00FBC7DB,2019-08-13T09:00,f,f,83514,19986,307200,614400,970523,172
 }}}

 There's one line per consensus entry, which is where we get the following
 columns from: `fingerprint,validafter,hasexitflag,hasguardflag,measured`.
 The `read` and `write` columns come from bandwidth histories contained in
 extra-info descriptors. `rate,burst,observed` come from the server
 descriptor referenced by the consensus entry.

 For CDF-Relay-Utilization I used (pseudo code):

 {{{
 read_write := (read + written) / 2;
 advertised := min(rate, burst, observed)
 plot -> advertised / read_write
 }}}

 For CDF-Relay-Stream-Capacity I used:

 {{{
 plot -> measured / observed
 }}}

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/33076#comment:8>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online