[tor-dev] Updated Proposal 324: RTT-based Congestion Control for Tor

[Mike Perry]

We have done another round of updates to Proposal 324:
https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/324-rtt-congestion-control.txt

Since the last update (summary provided below in the quoted reply), the
main changes are to include the drain rate of an edge connection in the
XON cell, for flow control. This reduces the XON/XOFF chatter and
average queue size in the event that edges are sending slower than their
Tor circuits.

Additionally, dropmark defenses were added in the form of controlling
the frequency of XON/XOFF, and in providing a time duration after which
we stop allowing data on half-closed connections. (As with all dropped
cell defenses in C-Tor, these are only enforced by the vanguards addon -
https://github.com/mikeperry-tor/vanguards/blob/master/README_TECHNICAL.md#the-bandguards-subsystem)

A C-Tor implementation of the flow control algorithm for edges has been
merged to tor.git's origin/main. It lives in
src/core/or/congestion_control_flow.[ch]. This implementation remains
off by default, pending implementation of protocol negotiation for
Congestion Control
(https://gitlab.torproject.org/tpo/core/tor/-/issues/40444).

This flow control implementation was evaluated and tuned by David Goulet
and myself over a custom onion service test bed, and we verified that it
is only active when edges are draining slower than their Tor circuit.

The specification has also been updated with additional details for what
remains to be tested and tuned in Shadow, as well as additional
consensus parameters external to congestion control that will require
re-tuning as well. These details can be found in Section 6 of Proposal
324. The consensus parameter list for tuning is in Section 6.5.

Here is the full Proposal 324 changelog since the last tor-dev post:
 - Export the monotime clock stall detection as a global property
 - Specify advertisement of edge drain rate in XON, to minimize chatter
 - Limit the frequency of XON/XOFF with consensus params
 - Describe oomkiller behavior wrt misbehaving endpoints
 - Describe dropmark defenses using XON/XOFF limits
 - Describe how half-closed edge connections are handled with flow
   control
 - Describe onion service advertisement and negotiation of congestion
   control
 - Describe flow control consensus parameters
 - Describe flow control shadow experiments and live comparison
 - Create and describe additional consensus parameters that will
   influence congestion control performance and memory usage
 - Clarify performance metrics involved in experiments
 - Describe more shortcomings of TOR_WESTWOOD/BOOTLEG_RTT_TOR
 - Remove some stale XXXs and TODOs

Review and comments are welcome!

On 7/30/21 6:22 PM, Mike Perry wrote:
> As a heads up, I have updated Proposal 324: RTT-based Congestion Control
> for Tor -
> https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/324-rtt-congestion-control.txt
> 
> The updated version has a new congestion control algorithm, detailed
> specifics on how to accurately measure RTT and BDP, and descriptions of
> algorithm parameter values and tuning experiments, among other improvements.
> 
> A C implementation of these algorithms has been merged to tor.git
> orgin/main. They live in src/core/or/congestion_control_*.[ch]. These
> algorithms are not enabled yet; we still need to implement flow control
> and negotiation, as well as do shadow experimentation to determine good
> candidate parameter values to further tune on live.
> 
> Here is a full changelog since the proposal was last published to tor-dev:
>  - Correct algorithms to update once per congestion window
>  - Add orconn blocking and edge connection checks as signals
>  - Specify BDP estimators based on onion service testing and eval.
>  - Specify a pure BDP tracking congestion control algorithm (TOR_NOLA)
>  - Update consensus parameters with tuning notes
>  - Document what we learned so far in live onion service experimentation
>  - Mention that we need to test intermittent downloads in Shadow/live
>  - Describe clock jump and stall detection during RTT measurement
>  - Mention if we calculate package window, it can become negative
>  - Break off the backwards ECN idea into a future idea draft:
> 
> https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/ideas/xxx-backward-ecn.txt
>  - Update old trac URLs
>  - Add Acknowledgements
> 
> Review and comments are welcome!
> 
> --
> Mike Perry
> _______________________________________________
> tor-dev mailing list
> tor-dev at lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
> 

-- 
Mike Perry