[tor-dev] Proposal: Expose raw bwauth votes

[Tom Ritter]

Updated proposal attached.

On 12 December 2017 at 12:44, isis agora lovecruft <isis at torproject.org> wrote:
>> Status: Open
>
> I changed things recently, you'll need a "Ticket:" field if your proposal is
> in state {OPEN,ACCEPTED,CLOSED,FINISHED}. [0]
>
> (Although, maybe we shouldn't require "Ticket:" for state OPEN, so as not to
> hinder calling it OPEN and discussing it even for those things which don't
> yet have tickets?)

I added Ticket: https://trac.torproject.org/projects/tor/ticket/21377


>> An authority SHOULD publish the bwauth vote used to calculate its
>> current vote. It should make the bwauth vote file available at the
>> same time as its normal vote file. It should make the file available
>> at
>>   http://<hostname>/tor/status-vote/next/bwauth.z
>
> If it's "next", how is it possible to expose it at the same time as its vote
> which is based upon it?  Maybe we should change the URL to be "current"?

teor suggested 'now'?  I'll make it whichever you think it should be =)


>> The raw bwauth vote file does not [really: is not believed to] expose
>> any sensitive information.  All authorities currently make this
>> document public already, an example is at
>>   https://bwauth.ritter.vg/bwauth/bwscan.V3BandwidthsFile
>
> Maybe we want to think about resource exhaustion attacks if we're making a
> standarised interface available to it?  The response after all is going
> likely always be much larger than the request.

teor suggested compressing and streaming from disk?

-tom
-------------- next part --------------
Filename: xxx-expose-bwauth_votes.txt
Title: Have Directory Authorities expose raw bwauth vote documents
Author: Tom Ritter
Created: 11-December-2017
Status: Open
Ticket: https://trac.torproject.org/projects/tor/ticket/21377

1. Introduction

Bandwidth Authorities (bwauths) perform scanning of the Tor Network
and calculate observed speeds for each relay. They produce a 'bwauth
vote file' that is given to a Directory Authority. The Directory
Authority uses the speed value from this file in its vote file
denoting its view of the speed of the relay.

After collecting all of the votes from other Authorities, a consensus
is calculated, and the consensus's view of a relay's speed is
determined by choosing the low-median value of all the authorities'
values for each relay.

Only a single metric from the bwauth vote file is exposed by a 
Directory Authority's vote, however the original file contains
considerably more diagnostic information about how the bwauth arrives
at that measurement for that relay.

2. Motivation

The bwauth vote file contains more information than is exposed in the
overall vote file. This information is useful to debug anomalies in
relays' utilization and suspected bugs in the (decrepit) bwauth code.

Currently, all bwauths expose the raw vote file through various (non-
standard) means, and that file is downloaded (hourly) by a single person
(as long as his home internet connection and home server is working)
and archived (with a small amount of robustness.)  

It would be preferable to have this exposed in a standard manner.
Doing so would no longer require bwauths to run HTTP servers to expose
the file, no longer require them to take additional manual steps to
provide it, and would enable public consumption by any interested
parties.  We hope that Collector will begin archiving the files.

3. Specification

An authority SHOULD publish the bwauth vote used to calculate its
current vote. It SHOULD make the bwauth vote file available at all
times, and provide the file that it has most recently used for its
vote (even if the vote is not currently published.) It SHOULD make
the file available at
  http://<hostname>/tor/status-vote/now/bwauth-legacy.z

It MUST NOT attempt to send its bwauth vote file in a HTTP POST to
other authorities and it SHOULD NOT make bwauth vote files from other
authorities available.

Clients interested in consuming the document should download it when
votes are created. (For the existing Tor network, this is at HH:50,
or 50 minutes after each hour.)

4. Security Implications

The raw bwauth vote file does not [really: is not believed to] expose
any sensitive information.  All authorities currently make this
document public already, an example is at
  https://bwauth.ritter.vg/bwauth/bwscan.V3BandwidthsFile

5. Compatibility

Exposing the document presents no compatibility concerns.

The compatibility concern is with applications that want to consume
the document. The bwauth vote file has no specification, and has been
extended in ad-hoc ways. Applications that merely wish to archive the
document (e.g. Collector) won't have a problems. Applications that
want to parse it may encounter errors if a new (unexpected) field is
added, if a new format is specified and fields are removed, or
assumptions are made about the text encoding or formatting of the
document.