[tor-bugs] #33871 [Core Tor/sbws]: Scale exactly as torflow does?

[Tor Bug Tracker & Wiki]

#33871: Scale exactly as torflow does?
-------------------------------------------------+-------------------------
 Reporter:  juga                                 |          Owner:  juga
     Type:  defect                               |         Status:
                                                 |  assigned
 Priority:  Medium                               |      Milestone:  sbws:
                                                 |  1.1.x-final
Component:  Core Tor/sbws                        |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  sbws-majority-blocker, sbws-roadmap  |  Actual Points:
Parent ID:  #33775                               |         Points:
 Reviewer:                                       |        Sponsor:
-------------------------------------------------+-------------------------
Changes (by juga):

 * cc: mike (added)


Comment:

 I'm CCing mike, cause i'd love he checks my interpretation on what Torflow
 does is correct, to be able to deploy sbws in all bwauths.

 Initially, i thought this ticket would be easy to solve by just taking the
 maximum between the descriptor bandwidth and the consensus bandwidth,
 instead of the minimum, or not considering the consensus bandwidth at all.

 Then i realized that would be a big error, since:
 a. that would make the new consensus bandwidth to depend basically on what
 the relay says to have
 b. at some point we migth remove the initial bandwidth self-test from tor
 (#22453)

 So i went again through Torflow code and i realized about the following:

 1. Filtered bandwidth

 While my understanding on how it is calculated in Torflow is correct, it
 was not the implementation.
 In sbws it is not calculated the filtered bandwidth for *each* relay, but
 only the one for all the relays. I'll submit a patch for this. This does
 not affect too much to the scaling though.

 2. Descriptor bandwidth

 TorCtl takes as observed bandwidth the minimum of the observed and burst
 descriptor bandwidths [6].
 I observed that if the burst bandwidth is 0, then the descriptor bandwidth
 will be 1, no matter if there was observed bandwidth.
 sbws takes as observed bandwidth the minimum of all descriptor bandwidths
 since some version, except that is currently discarding any relay without
 descriptor average, not only any relay without observed bandwidth. I'll
 submit a patch for this.

 sbws has not being having descriptor or consensus bandwidth updated
 because of other bugs, which should be fixed now (to check in #33947).

 However, what should happen when sbws is missing one of the descriptor
 observed, average or burst bandwidth?
 Without the observed bandwidth, doesn't make sense to take the minimum of
 the other two, cause they're values set by the operator and are usually
 bigger.
 Would it be fine to take only the observed bandwidth when the average
 or/and burst bandwidth are missing? (assuming is not a sbws bug), or if
 the operator set either average or burst bandwidth to 0, should we still
 consider observed bandwidth to be 1?

 3. Descriptor bandwidth vs consensus bandwidth:

 In #28598 we started to take the minimum of the descriptor bandwidth and
 the consensus bandwidth.

 According to Torflow `README.spec.txt` [2]:

 {{{
 For the case where K_p = 1, K_i=0, and K_d=0, it can be seen that this
 system is equivalent to the one defined in 2.2, except using consensus
 bandwidth instead of descriptor bandwidth
 }}}

 In current Torflow code, the `K` variables have those values, but it's
 using the descriptor, not the consensus.
 In `aggregate.py`, `bwauth_pid_control` and `use_desc_bw` are initialized
 to `True` [3], so `Node` is also initialized to use the descriptor
 bandwidth and not the consensus bandwidth [4]. I'm quite sure that any
 other if-else where `use_bw` might be something else, it's node cause the
 default initialization makes that code is never executed.
 However Toflow does not seem to scale at all if there is a consensus
 bandwidth missing. [8]

 sbws takes only the descriptor bandwidth if the consensus bandwidth is
 missing.
 But what if the descriptor bandwidth is missing and consensus bandwidth is
 present?, should we take only the consensus bandwidth?.
 Assuming it's not an sbws bug, is it possible to have only consensus
 bandwidth without descriptor bandwidth?
 Should we scale in any case taking the minimum of the observed bandwidth
 and consensus bandwidth, which would be 1 if any of them is missing?

 Not having an easy/fast way to simulate what happen when we replace all
 torflows by sbws, we'd need anyways to human-monitor the consensus when we
 start having again the half of bwauths with sbws.

 [2]
 https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/README.spec.txt#n390
 [3]
 https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/aggregate.py#n237
 [4]
 https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/aggregate.py#n589
 [6] https://gitweb.torproject.org/pytorctl.git/tree/TorCtl.py#n460,
 https://gitweb.torproject.org/pytorctl.git/tree/TorCtl.py#n488
 [8]
 https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/aggregate.py#n402

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/33871#comment:9>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online