[tor-bugs] #9321 [Tor]: Load balance right when we have higher guard rotation periods

[Tor Bug Tracker & Wiki]

#9321: Load balance right when we have higher guard rotation periods
-------------------------+-------------------------------------------------
     Reporter:  arma     |      Owner:
         Type:  project  |     Status:  new
     Priority:  normal   |  Milestone:  Tor: 0.2.6.x-final
    Component:  Tor      |    Version:
   Resolution:           |   Keywords:  needs-proposal, tor-auth, tor-
Actual Points:           |  client, 026-triaged-1
       Points:           |  Parent ID:  #11480
-------------------------+-------------------------------------------------

Comment (by asn):

 OK, I have some ideas on how this might get deployed to directory
 authorities.

 I now have two scripts: One script (`summizer.py`) summarizes consensuses
 into compact summary files, and the next script (`guardiness.py`)
 calculates the guardiness of nodes given a bunch of summary files. In the
 end, `guardiness.py` outputs a file with the guardiness information that
 is to be read by little-t-tor before forming a vote.

 When a directory authority operator wants to start using the guardiness
 system (bootstrap), she makes a '''consensus directory''' and downloads 3
 months worth of consensuses in there (probably from metrics). Then she
 runs the `summizer.py` script which reads the consensuses from that
 directory, and saves a summary file for each consensus in a separate
 '''summary file directory'''. This step is expected to take 30 minutes or
 so.

 Then the dirauth operator will add a cron job, similar to the one for bw
 auths, that triggers in the 35th minute of each hour (or something). This
 cron job does the following:
 * Download latest consensus from metrics.
 * Run `summizer.py` to summarize the new consensus (This should be instant
 since its just one consensus).
 * Run `guardiness.py 3 months` to calculate guardiness of all summary
 files in the summary directory from the past 3 months (This will take
 between a few seconds to 2 minutes of intense computation).

 Then 5 minutes before the hour, little-t-tor will read the guardiness
 output file and consider its data when voting.

 Some notes:

 * Instead of downloading the latest consensus from metrics, maybe we could
 add little-t-tor code that would save new consensuses into a directory.
 This way, we don't need to download bulk from metrics, apart from during
 the bootstrap procedure.

 * To avoid keeping old summary/consensus files around that take over disk
 space, I added some optional cleanup switches to the scripts.
 Specifically, the `summizer.py` script can delete the consensus files that
 got summarized and can also delete consensus files older than 3 months (or
 N months). Similarly, the `guardiness.py` script can delete summary files
 older than 3 months (or N months). The idea is that every time the cron
 job triggers, the `summizer.py` and `guardiness.py` scripts will auto-
 delete the oldest summary/consensus file, keeping in disk only the useful
 files.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9321#comment:22>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online