[tor-project] Bandwidth scanner report - October-December

[juga]

Hi,

The main news during October-December were:
 - sbws 1.0 was released in October, including a Debian package and
OpenBSD package. However it's recommended that bandwidth authorities
wait for 1.0.3 release to start running it.
 - longclaw started to use bandwidth files created by sbws in November.
This allowed us to detect more bugs. It has been stable since December.

In October we decided that the short term main goals are:
- Get sbws to scale and produce similar results as torflow.
- Change Torflow to sbws in the current bandwidth directories .
- Get all the directory authorities to run sbws.
In a longer term, when all torflow instances are replaced, the main goal
is to decide which should be the load balance in the Tor network.

These were most of the tasks:
* New features:
  - Add documentation about how the bandwidth measurements are selected
and scaled before writing them to the bandwidth file.
  - Warn when the difference between the total consensus bandwidth and
the total bandwidth with the relays included in the bandwidth file is
larger than 50%.
  - Include headers in the bandwidth file about the percentages of
measured relays that are included in the file.
  - Warn when the percentage of relays in the bandwidth file is less
than the 60% of the relays in the network.
  - Write bandwidth file only when the percentage of measured relays is
bigger than 60%.
  - Change the name of some keys used in the bandwidth file.
  - Do not measure round-trip time by default. It was proven that
latency didn't affect much the measurements, and was slowing down the
scanner.
  - When scaling bandwidths as torflow, round to 2 significant figures
by default. This implements part of proposal 276.
  - Change Python minimal version.
  - Change default directories when sbws is run from a system service.
  - Do not overwrite bandwidth file to avoid race conditions, instead
generate the files atomically.
  - Add command to remove old bandwidth files.
  - Create a Debian package.
  - Update sbws to use stem 1.7.0 release.
  - Add a command line argument for the configuration file.
  - Improve INSTALL and DEPLOY documentation.
  - Update bandwidth file specification.

* Fixed bugs:
  - To include a relay in the bandwidth file, at least 2 measurements
for that relay must be X seconds away.
  - Use the option to take into account measurements that are away from
each other X seconds when it's enabled .
  - Do not resolve destination domain locally and unify criteria to
choose relays as exits.
  - Bandwidth filtered is the maximum between the bandwidth measurements
and their mean, not the minimum.
  - Stop measuring the same relay by two threads when a new loop with
the new relays to measure starts.
  - Ensure de default scaling method is torflow, leave command line
argument option but ignore it, update documentation reflecting this.
  - Continue running with default configuration when the config command
line argument argument is provided but the file can not be found.
  - Use 5 "=" characters as terminator in the bandwidth files.
  - Update bandwidth file specification version in the `generator`.
  - Compare with consensus bandwidth and MaxAdvertisedBandwidth when
scaling as torflow.
  - Remove old links in the documentation.


Cheers!,
juga.