[metrics-bugs] #24729 [Metrics/Onionoo]: Find reason for 'null' values in Onionoo document

[Tor Bug Tracker & Wiki]

#24729: Find reason for 'null' values in Onionoo document
-----------------------------+------------------------------
 Reporter:  Dbryrtfbcbhgf    |          Owner:  metrics-team
     Type:  defect           |         Status:  new
 Priority:  High             |      Milestone:
Component:  Metrics/Onionoo  |        Version:
 Severity:  Major            |     Resolution:
 Keywords:                   |  Actual Points:
Parent ID:                   |         Points:
 Reviewer:                   |        Sponsor:
-----------------------------+------------------------------

Comment (by karsten):

 Here's what happened:
  - The relay upgraded from a tor software version reporting bandwidth for
 4 hour intervals to 0.3.2.6-alpha on December 2 and later downgraded to a
 pre-0.3.2.6-alpha version on December 21. During that time it reported
 bandwidth values for 24 hour intervals.
  - Onionoo's 1 month graph has a fixed data point interval of 4 hours. It
 cannot report data with a resolution of only 1 data point per 24 hours.
 That's why Onionoo includes all those `null` values, simply meaning that
 it doesn't know how much bandwidth the relay read or wrote during that
 time.
  - Relay search displays those `null` values as `0` values, which it
 shouldn't do. Knowing that a relay pushed 0 bytes is a different thing
 than not knowing how many bytes it pushed.
  - Only the 1 month and 3 months graphs are affected, because they have a
 data resolution of 4 hours and 12 hours, respectively. Look at the 1 year
 graph with a data resolution of 48 hours and which doesn't have a gap in
 December 2017.
  - Note that this is the same issue that leads to blank 1 month and 3
 month graphs and non-blank 1 year graphs for new relays.

 Is this a bug in Onionoo? Probably.

 What can we do? We could go through all reported data points for a given
 graph and pick the largest interval as data point interval. That is, if
 all data points have an interval of 4 hours, we produce graph data with 1
 data point per 4 hours. But if there's at least 1 reported data point with
 an interval of 24 hours, we aggregate all data for that graph to 1 data
 point per 24 hours.

 A possible downside is that the graphs for 3 months, 1 month, 1 week, and
 3 days will basically contain the exact same data. Maybe we can do
 something smart to avoid providing data that is too redundant, but not too
 smart in order to not move too much logic into clients.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/24729#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online