Proposal 168: Reduce default circuit window

[Karsten Loesing]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Björn,

I'm commenting on just a few points here:

On 09/01/2009 01:50 PM, Björn Scheuermann wrote:
> this sounds like an interesting proposal! However, I'd like to point you to 
> some implications and side effects. Over the past months, one of my students 
> (Daniel Marks, on CC) has worked on congestion control in Tor. One of his 
> results is an implementation of Tor circuits in a network simulation framework 
> (ns-2). We use this implementation to simulate Tor networks of different size 
> and with different traffic patterns, play with various parameters and measure 
> throughput, cell latencies, etc. Currently, our aim is to gain a better 
> understanding of congestion effects in Tor and their implications and 
> interrelations.

Sounds great. Are you planning to release your simulation sources? I'd
lie when saying that I want to look at the sources now, but I certainly
would like to play with your simulation at a later point. As may others.

>> 2. Motivation
>>
>>   Karsten's torperf graphs show that the median download time for a 50KB
>>   file over Tor in mid 2009 is 7.7 seconds, whereas the median download
>>   time for 1MB and 5MB are around 50s and 150s respectively. The 7.7
>>   second figure is way too high, whereas the 50s and 150s figures are
>>   surprisingly low.
> 
> Your statement implies that you are looking at something like the 
> "transmission latency per KB" (which is, by the way, identical to the inverse 
> throughput).

True, most of the torperf graphs are for measuring bandwidth rather than
latency. See the PDF reports for more information:

https://git.torproject.org/checkout/metrics/master/report/performance/torperf-2009-08-24.pdf

https://git.torproject.org/checkout/metrics/master/report/circwindow/circwindow-2009-08-19.pdf

The early results of cells in circuit queues might help, too. Note that
more work remains to make real use of these data. Want to help? :)

https://git.torproject.org/checkout/metrics/master/report/buffer/bufferstats-2009-08-25.pdf

>>   To get a more concrete sense of the benefit, though, Karsten has been
>>   running torperf side-by-side on exit relays with the old package window
>>   vs the new one. The results are mixed currently -- it is slightly faster
>>   for fetching 40KB files, and slightly slower for fetching 50KB files.
> 
> Note that file sizes of 40 or 50 KB cannot tell you much about such a 
> modification. As you state above, 100 cells is equivalent to ~50 KB of payload. 
> I.e., you do the whole transmission within the initial transfer window. So, 
> the data transfer will be over before the circuit's window mechanisms have had 
> a chance to become active at all! Consequently, modifications to the window 
> mechanism will not impact transfers that are below that threshold directly.

The 50 KiB (kibibytes, not kilobytes) download should not fit into one
circuit window of 101 cells. That circuit window can hold up to 101 x
498 = 50298 bytes whereas the file size is 51200 bytes. Due to the
torperf output, the client receives even 51466 bytes. So, we need a
second circuit window to transfer the 50 KiB file.

But I admit that 50 KiB is still pretty low. Right now, we have more
measurements running with 1 MiB files. They just take somewhat longer to
complete, because we cannot start new downloads in 5-minute intervals,
but only in 30-minute intervals. At the moment we have 347 data points;
not enough to make a useful statement. It's going to take another two
weeks to have enough data.

>>   I think it's going to be tough to get a clear conclusion that this is
>>   a good design just by comparing one exit relay running the patch. The
>>   trouble is that the other hops in the circuits are still getting bogged
>>   down by other clients introducing too much traffic into the network.
> 
> Maybe I'm getting something royally wrong - but what exactly are the 
> mechanisms within an onion router that would result in one circuit 
> experiencing much longer cell queuing delays just because *another* circuit 
> has a long queue in that router? In terms of fair resource sharing between 
> circuits (and the opportunity to forward cells quickly), our impression is 
> that the round-robin scheduler does its job pretty well. It will be hard to 
> improve on that just by introducing intentional resource underutilization.
> But you guys know the Tor sources much better than we do - are we missing 
> something important here?

The problem of loud circuits slowing down other circuits emerges when
these circuits are sharing the same TCP connection.

Did you have a look at Joel's and Ian's USENIX paper?

"All traffic between any pair of routers, even if they represent
circuits for different clients, are multiplexed over a single TCP
connection. This results in interference across circuits during
congestion control, packet dropping and packet reordering. This
interference greatly contributes to Tor’s notorious latency problems."

See http://crysp.uwaterloo.ca/publications/ for the publication list or
http://www.cypherpunks.ca/~iang/pubs/TorTP.pdf for the PDF.

Best,
- --Karsten

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkqnfPgACgkQ0M+WPffBEmVF0ACgzroWeSx5CMtNJ9kHJDx7yXpI
/DwAoLrR4HJN606+zBZbheBFUzfUvuuA
=Ss05
-----END PGP SIGNATURE-----