[metrics-bugs] #21378 [Metrics/CollecTor]: Archive bwauth bandwidth files

[Tor Bug Tracker & Wiki]

#21378: Archive bwauth bandwidth files
------------------------------------+------------------------------
 Reporter:  tom                     |          Owner:  metrics-team
     Type:  enhancement             |         Status:  new
 Priority:  Medium                  |      Milestone:
Component:  Metrics/CollecTor       |        Version:
 Severity:  Normal                  |     Resolution:
 Keywords:  tor-bwauth tor-dirauth  |  Actual Points:
Parent ID:                          |         Points:
 Reviewer:                          |        Sponsor:
------------------------------------+------------------------------

Comment (by karsten):

 Replying just to the parts where I wouldn't reply "okay, cool!":

 Replying to [comment:15 teor]:
 > Replying to [comment:14 karsten]:
 > > In the future, we can switch to fetching only those files that are
 referenced from votes, unless for some reason we want to have non-
 referenced files, too.
 >
 > How are bandwidth files referenced from votes?
 >
 > I don't think we will implement "bandwidth-file-url" from
 https://trac.torproject.org/projects/tor/ticket/21378?replyto=14#comment:6
 >
 > Tor 0.3.5? and later add bandwidth file headers to each vote, and we may
 add a bandwidth file hash in future. Once all authorities upgrade, you can
 fetch the bandwidth file if the vote contains headers.

 I guess I was thinking of 0.3.5 then. I'm not aware of any other plans.

 > >   - The relaydescs module runs twice per hour, so it's going to
 download the file twice every hour. Again, if we only fetch referenced
 files, we wouldn't download the same file more than once.
 >
 > I am not sure if we plan on implementing "referenced files" in Tor. Can
 you explain what you mean?

 Same as above: bandwidth files referenced from votes.

 > >   - I assume there are no plans that authorities serve bandwidth files
 of other authorities? That's different for votes which are cached by other
 authorities. Should be fine, but something to consider for the future.
 >
 > Votes are posted, fetched, and cached by authorities so that each
 authority can create a consensus.
 > There's no equivalent for bandwidth files, so we probably won't
 implement bandwidth file caching.
 > But if you tell us you really need it, we could work something out.

 Sounds reasonable. I don't think we'll need it.

 > > > 2) Teach
 [https://gitweb.torproject.org/collector.git/tree/src/main/java/org/torproject/collector/relaydescs/RelayDescriptorParser.java
 RelayDescriptorParser] to recognise the file
 > >
 > >  - While we're waiting for #21377, can we have a sample file to start
 writing some parsing code?
 >
 > moria1's votes have a bandwidth-file-headers line, but it's based on
 torflow's bandwidth file, so it only has a timestamp:
 > http://128.31.0.34:9131/tor/status-vote/current/authority
 >
 > The bandwidth file spec contains sample data:
 > https://gitweb.torproject.org/torspec.git/tree/bandwidth-file-
 spec.txt#n450
 >
 > We're working on version 1.2.0 of the format for sbws 1.0 in #28085.
 When sbws 1.0 is ready, we will update the spec with sample data from the
 latest sbws.

 Thanks! I didn't look just yet, but this should be a good start to write
 some code.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21378#comment:16>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online