[tor-dev] Bandwidth Authority Progress

[teor]

On 20 Dec 2017, at 06:06, meejah <meejah at meejah.ca> wrote:

>> This project is not quite there yet, and will require some
>>   non-trivial engineering time, but it's probably a much easier task
>>   compared to peerflow due to the design being more understood and
>>   already coded.
> 
> I'm not convinced this part is completely accurate ;) because at TorDev
> MTL it seems to me the consensus was that nobody actually knows what
> torflow is doing and so answering the question "is bwscanner doing the
> same thing" is approximately NP-hard.

I have some idea what torflow is doing, in a broad sense:
* launch 2 tor clients
* repeat as often as possible, running 9 different scanners:
    * split relays into buckets by bandwidth percentile
    * build two hop paths with a relay and exit from relays in the bucket
    * download a file from a bandwidth server, choose the size based on the bucket
    * measure how long it takes
    * store the results in a database
* aggregate the results hourly:
    * produce a consensus weight to advertised bandwidth ratio
    * using a decaying weighted average
    * and some form of feedback (PID) control
    * and dump it to a file
Then authorities read this file and include it in their votes.

I suspect that Mike Perry may remember more detail, or may want to
correct my summary, as he wrote most of torflow (I think?)

My conclusion at the Montreal meeting was that we don't have a
detailed spec (see below). So that makes it hard to tell if:
* torflow does what we want it to do
* the new bwauth project does what we want it to do
* they are similar enough for a staged or once-off transition

>> I think 2-3 weeks of developer time could be quite fruitful here. I
>> also heard that some bw auth operators are eager to run bwscanner
>> instead of torflow on their setup in January.
> 
> Wooo!
> (I think the best path to answering "does bwscanner do the same thing as
> torflow" is to Run It And See...) If any of these parties are having
> problems deploying bwscanner this is probably something I can help with.

It doesn't produce an output file in the same format as torflow, so we need
to specify (see below) and implement that part first.

Otherwise, we would not have any results to compare.

>> Currently my intuition is to work on (b) above, while also preparing the
>> ground for (a) which seems to be The Right Thing.
> 
> +1
> I think the next step for a) isn't "implement it", but "write a spec for
> it" instead.

+1

Let's start by specifying what tor directory authorities expect from the
file format.

T