[tor-dev] Bandwidth Authority Progress

[George Kadianakis]

teor <teor2345 at gmail.com> writes:

> Hi asn,
>
> Original Subject: Re: [tor-project] Meeting notes, network team meeting, 18 Dec
>
>> On 19 Dec 2017, at 07:28, Nick Mathewson <nickm at torproject.org> wrote:
>> 
>>  - Met with David Stainton, Moritz and others. Talked about relay load
>>    balancing and bandwidth dirauths. People are sad about the state of the
>>    network: some relays are overloaded while others idle, many overloaded
>>    relays cant even establish circuits to each other. Need to do something
>>    about it: deploy bwscanner and start thinking about peerflow. What about
>>    isis' bridge bandwidth scanner?
>
> Can you tell us a bit more about this meeting?
> What can people do if they want to be involved?
> When is the next meeting, or how can people find out about it?
>

Hey teor,

thanks for following through.

The "meeting" was impromptu and IRL because we all happened to be at the
same place. There is no next meeting and it's up to us (Tor/network
team) to figure out what are the next steps here.

This week I did some digging to explore the various possible ways
forward also based on your email here: https://www.mail-archive.com/tor-dev@lists.torproject.org/msg09912.html
Here are my findings:

Possibility a) Develop peerflow and deploy it in place of torflow

   Peerflow is an exciting and secure bandwidth measurement system
   published in PETS 2017: https://ohmygodel.com/publications/peerflow-popets2017.pdf

   Unfortunately, it seems quite complicated to develop from scratch and
   will probably require _significant_ engineering time to actually make
   it a deployed reality (understand, develop, test, deploy). This is
   probably the solution we would like to pursue if we had a grant and a
   dedicated developer.

Possibility b) Finalize bwscanner and deploy in place of torflow

   bwscanner is a project by Aaron/David/Donncha: and can be found here:
   https://github.com/TheTorProject/bwscanner

   It seems to implement the torflow design (2-hop circs && buckets) but
   in a cleaner and better codebase. From what I understand, the main
   part of the project is done, but there has been minimal testing on
   the real network (there are unittests tho) and also the final output
   file with the bandwidth weights has not been completely finalized.

   This project is not quite there yet, and will require some
   non-trivial engineering time, but it's probably a much easier task
   compared to peerflow due to the design being more understood and
   already coded. I think 2-3 weeks of developer time could be quite
   fruitful here. I also heard that some bw auth operators are eager to
   run bwscanner instead of torflow on their setup in January.

Possibility c) Adapt the bridge bw scanner that is currently being developed

   Apparently isis and another developer are currently writing a bridge
   bandwidth scanner for bridgedb, that could in theory be extended to
   scan the whole network. They are currently writing some sort of Rust
   library that will be used by the scanner, and the project is ETA
   around March 2018. The whole development process is pretty opaque so
   I have no idea what's going on. Also, there probably needs to be
   considerable work to extend it from a simple bridge scanner to a real
   relay scanner, and the final result will probably look like bwscanner
   above.

Currently my intuition is to work on (b) above, while also preparing the
ground for (a) which seems to be The Right Thing.

I'm not sure what's the right way forward here in terms of project
management, since the network team seems overloaded and I haven't heard
of anyone willing to take this on...

Ideally we would probably apply for some sort of grant on this work so
that some actual developer time is allocated. I think this is definitely
fundable work since it deeply impacts the *performance* and security of
the Tor network, and basically the network has no chance of surviving in
greater loads if the status quo persists.

I'll try to think more about this problem in the future, these are just
my thoughts from a few hours of digging.  

Cheers!