[tor-dev] Proposal Waterfilling

[Florentin Rochet]

Hi teor,

Sorry about the huge delay :)

I've added your following idea to the proposal (seems we come up to the
right way to do it :-)):

> Why not list the waterfilling level on a single line in the consensus?
>
> That way:
> * authorities do the expensive calculation
> * clients can re-weight relays using a simple calculation:
>
> if it is less than or equal to the waterfilling level:
>   use the relay's weight as its guard weight
>   use 0 as its middle weight
> otherwise:
>   use the waterfilling level as the relay's guard weight
>   use the relay's weight minus the waterfilling level as its middle weight
>
> This is O(n) and requires one comparison and one subtraction in the worst case.

I have a few questions about convergence/divergence of weights, but
maybe we could take advantage of the meeting in Rome to discuss this avenue?

On 28/01/18 23:40, teor wrote:
> <skip>
> I'm going to re-ask this questions, in light of the extra middle load from
> Tor2web clients:
>
> Does the waterfilling proposal make excessive load on middles worse, by
> allocating more middle weight to higher capacity relays?
>

If bwauths overestimate top relays, or if we reach some soft limit, yes.
But the reverse would be true too: if we have excessive load of guards,
then this proposal will make things better.

> In particular, connections are limited by file descriptors, and file
> descriptor
> limits typically don't scale with the bandwidth of the relay. As far
> as I can tell,
> waterfilling would have directed additional Tor2web traffic to large
> guards.
> It would have brought down my guards faster, and made it much harder
> for me
> to keep them up.
>
> If we had implemented waterfilling before this attack, would it have
> lead to
> cascading failures on our top guards? They would have been carrying
> significantly more middle load, and mine barely managed to cope.
>

Probably yes, but they would also carrying less load at the guard
position from normal Tor users. In normal condition, that should tie. In
a DDoS situation, I would say we face difficulties no matter what we do.

> Can you redesign the proposal so there is some limit on the extra
> middle load
> assigned to a guard? Or does this ruin the security properties?
>
> Is there a compelling argument for security over network robustness?
>

Questions added to the proposal :)

> <skip>
>> Could you be
>> more specific about your concerns with the bandwidth authorities and
>> this proposal?
>
> It takes time and effort from Tor people to integrate and maintain the
> code and monitoring for a new proposal like this one.
>
> We will need to take extra time on this proposal, because we already need
> more monitoring for the current bandwidth authority system. And only then
> would we have time to build monitoring specific to this proposal.
>
> Also, when we change bandwidth measurement or allocation, we need to
> change one thing at a time, and then monitor the change. So depending
> on our priorities, this proposal may need to wait until after we implement
> and monitor other urgent fixes.
>

Yes, this proposal can wait, it's totally up to you. I agree that fixing
the current bwauthorities should be somewhat on the top of priorities.
But nothing prevents us to further discuss this proposal and make plans.

>>> Who will maintain the new code we add to Tor to implement waterfilling?
>>
>> I would volunteer to that.
>
> Typically, experienced Core Tor team members review and maintain code.
>
> And there's still a lot of development and testing work to be done before
> the code is ready to merge. Are you able to do this development?
>

If I can make this as a research project, that can be very fast. If I
have to do this on my spare time, that's going to be a bit slower.

> How much help will you need to write a new consensus method?
> How much help will you need to write unit tests?
> (This help will come from existing team members.)
>

I should be fine with both. Adding a new consensus method does not seem
too much difficult at first glance, but removing one looks a bit more
challenging. Hopefully, I just need to add one :)

> Does your current code pass:
> * make check
> * make test-network-all
>   * in particular, any new consensus method must pass the "mixed" network,
>     with an unpatched Tor version in your path as "tor-stable"
>

As much as I recall, make check passed as well as chutney networks. I
don't remember what test I did back at that time, though. If we go for
an up-to-date implementation, I willl make sure everything's ok :)

> <skip>
>> Bandwidth-weights and measurements (consensus weights) are two different
>> things that solve 2 different problems. So, we can work independently on
>> improving measurements (like what is currently done with bwscanner) and
>> improving Tor's balancing (bandwidth-weights) with this proposal.
>
> I don't think this is realistic.
> There is always contention for shared resources.
>
> Integrating and testing new code, and monitoring its effects, will
> take effort from
> the teams I mentioned above. This takes away from the urgent work of
> fixing the
> bandwidth authority system. Which also takes effort from the Core Tor and
> Metrics teams.

Right, totally agree that the first focus should be on bwauths. Could we
try to make plan, or at least move this proposal into the todo list?
Looking forward to meet you in Rome :)

Best,
Florentin