[metrics-bugs] #30218 [Metrics/CollecTor]: Add bandwidth files archiving to CollecTor

[Tor Bug Tracker & Wiki]

#30218: Add bandwidth files archiving to CollecTor
-------------------------------------------------+-------------------------
 Reporter:  irl                                  |          Owner:
                                                 |  metrics-team
     Type:  enhancement                          |         Status:  new
 Priority:  Medium                               |      Milestone:
Component:  Metrics/CollecTor                    |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  tor-bwauth,tor-dirauth,metrics-      |  Actual Points:
  roadmap-2019-q2                                |
Parent ID:  #21378                               |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------

Comment (by karsten):

 Yesterday, I wrote a little script that ran roughly once per minute for
 over an hour and fetched moria1's and longclaw's "next" bandwidth files. I
 received a bandwidth file every time, not just between, say, HH:49 and
 HH:00. More specifically, here's what I got:

 || '''Authority''' || '''Timestamp''' ||'''Digest''' || '''First
 received''' || '''Last received''' || '''Referenced from vote''' ||
 || longclaw || 1555868103 (17:35) ||`lKTscsfb..` || .. || 18:40 || 18:00
 ||
 || longclaw || 1555871704 (18:35) ||`8KuO5fcL..` || 18:43 || 19:40 ||
 19:00 ||
 || longclaw || 1555875303 (19:35) ||`laoWH3KH..` || 19:41 || .. || 20:00
 ||
 || moria1 || 1555867341 (17:22) ||`EAiqle6R..` || ..  || 18:45 || 18:00 ||
 || moria1 || 1555871524 (18:32) ||`5aZPyxPy..` || 18:46 || 19:45 || 19:00
 ||
 || moria1 || 1555875627 (19:40) ||`ZTrHiTtI..` || 19:46 || .. || 20:00 ||

 Looking at the 19:00 votes, we could fetch referenced bandwidth files from
 around 18:45 to around 19:45.

 For reference, we're currently fetching votes (and all other relay
 descriptors) starting at HH:05 and once more at HH:35 in case there was no
 consensus at HH:00.

 How about we simply download "next" bandwidth files at around HH:05 and at
 around HH:35, knowing that we're really going to receive "previous"
 bandwidth files? This would be easiest with regard to extending
 CollecTor's relaydescs module.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/30218#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online