[tor-dev] Proposal: Expose raw bwauth votes

[teor]

Hi,

It looks like this proposal was lost, and never committed to torspec.
Sorry about that!

Juga and I have updated the proposal.
Please see the full text inline below.

We have also written the corresponding dir-spec patch:
https://github.com/torproject/torspec/pull/26/files

We are tracking the spec and proposal in:
https://trac.torproject.org/projects/tor/ticket/26694

The Tor implementation in:
https://trac.torproject.org/projects/tor/ticket/21377

And the Collector implementation in:
https://trac.torproject.org/projects/tor/ticket/21378

> On 16 Jan 2018, at 08:23, Tom Ritter <tom at ritter.vg> wrote:
> 
> On 12 December 2017 at 12:44, isis agora lovecruft <isis at torproject.org> wrote:
> 
>>> An authority SHOULD publish the bwauth vote used to calculate its
>>> current vote. It should make the bwauth vote file available at the
>>> same time as its normal vote file. It should make the file available
>>> at
>>>  http://<hostname>/tor/status-vote/next/bwauth.z
>> 
>> If it's "next", how is it possible to expose it at the same time as its vote
>> which is based upon it?  Maybe we should change the URL to be "current"?
> 
> teor suggested 'now'?  I'll make it whichever you think it should be =)

We changed the scheme so the bandwidth file at
   status-vote/(next|current)/bandwidth.z

is the bandwidth file used to produce:
   status-vote/(next|current)/authority.z

We publish the bandwidth file to next/ when the vote is published to next/.
And the same with current.

It should be relatively easy to implement this publishing in the tor code.
And to implement downloads in Collector.

>>> The raw bwauth vote file does not [really: is not believed to] expose
>>> any sensitive information.  All authorities currently make this
>>> document public already, an example is at
>>>  https://bwauth.ritter.vg/bwauth/bwscan.V3BandwidthsFile
>> 
>> Maybe we want to think about resource exhaustion attacks if we're making a
>> standarised interface available to it?  The response after all is going
>> likely always be much larger than the request.
> 
> teor suggested compressing and streaming from disk?

We’ll need to take a copy of the file anyway, to avoid race conditions.
We can use the same code we use for the rest of the directory documents.

T

Filename: xxx-expose-bandwidth-files.txt
Title: Have Directory Authorities expose raw bandwidth list files
Author: Tom Ritter
Created: 11-December-2017
Status: Open
Ticket: https://trac.torproject.org/projects/tor/ticket/21377

1. Introduction

Bandwidth Authorities (bwauths) perform scanning of the Tor Network
and calculate observed bandwidths for each relay. They produce a bandwidth
list file that is given to a Directory Authority. The Directory
Authority uses the bw (bandwidth) value from this file in its vote file
denoting its view of the bandwidth of the relay.

After collecting all of the votes from other Authorities, a consensus
is calculated, and the consensus's view of a relay's speed is
determined by choosing the low-median value of all the authorities'
values for each relay.

Only a single metric from the bandwidth list file is exposed by a
Directory Authority's vote, however the original file contains
considerably more diagnostic information about how the bwauth arrives
at that measurement for that relay.

For more details, see the bandwidth list file specification in
bandwidth-file-spec.txt.

2. Motivation

The bandwidth list file contains more information than is exposed in the
overall vote file. This information is useful to debug:
  * anomalies in relays' utilization,
  * suspected bugs in the (decrepit) bwauth code, and
  * the transition to a replacement bwauth implementation.

Currently, all bwauths expose the bandwidth list file through various (non-
standard) means, and that file is downloaded (hourly) by a single person
(as long as his home internet connection and home server is working)
and archived (with a small amount of robustness.)

It would be preferable to have this exposed in a standard manner.
Doing so would no longer require bwauths to run HTTP servers to expose
the file, no longer require them to take additional manual steps to
provide it, and would enable public consumption by any interested
parties.  We hope that Collector will begin archiving the files.

3. Specification

An authority SHOULD publish the bandwidth list file used to calculate its
next vote. It SHOULD make the bandwidth list file available whenever the
corresponding vote is available, at the corresponding URL. (See
dir-spec for the exact details.)

It SHOULD make the file available at
  http://<hostname>/tor/status-vote/next/bandwidth.z
  http://<hostname>/tor/status-vote/current/bandwidth.z

It MUST NOT attempt to send its bandwidth list file in a HTTP POST to
other authorities and it SHOULD NOT make bandwidth list files from other
authorities available.

Clients interested in consuming these documents should download them from
each authority's:
  * next URL when votes are created. (In the public Tor network, this is
    after HH:50 during normal operation, and after HH:20 during a
    consensus failure.)
  * current URL after the valid-after time in the consensus.
    (After HH:00, and HH:30 during consensus failure.)

4. Security Implications

The raw bandwidth list file does not [really: is not believed to] expose
any sensitive information.  All authorities currently make this
document public already, an example is at
  https://bwauth.ritter.vg/bwauth/bwscan.V3BandwidthsFile

5. Compatibility

Exposing the document presents no compatibility concerns.

Applications that parse the document should follow the bandwidth list file
specification in bandwidth-file-spec.txt.
If a new bandwidth list format version is added, the applications MAY need
to upgrade to that version.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20180716/dc124833/attachment.sig>