[tor-dev] Proposal 213: Remove stream-level sendmes from the design

[Roger Dingledine]

On Mon, Nov 05, 2012 at 08:31:05AM +0100, Andreas Krey wrote:
> 'Receiver of the stream does not read anymore, for whatever reason.'
> 
> > With no per-stream window, the circuit will round-robin between the
> > streams that want to send a cell, just as it does now.
> 
> But where does the data go when the end (socks client or the server the
> exit node is talking to) isn't accepting any more (and tor can't write
> to the TCP socket)?
> 
> The sending side is pushing stream data into the circuit, and the
> receiving side (tor process) must either collect it locally (thereby
> growing the memory footprint), or not allow new circuit window, thereby
> affecting the other streams on the circuit.
> 
> Or did I miss another per-stream feedback/flow control mechanism?

Ah ha!

Yes, I think you're right.

Looking at it from the exit relay's perspective (which is where it matters
most, since most use of Tor is sending a little bit and receiving a lot):
when a create cell shows up to establish a circuit, that circuit is
allowed to send back at most 1000 cells. When a begin relay cell shows
up to ask that circuit to open a new stream, that stream is allowed to
send back at most 500 cells.

Whenever the Tor client has received 100 cells on that circuit, she
immediately sends a circuit-level sendme back towards the exit, to let
it know to increment its "number of cells it's allowed to send on the
circuit" by 100.

However, a stream-level sendme is only sent when both a) the Tor client
has received 50 cells on a particular stream, *and* b) the application
that initiated the stream is willing to accept more data.

If we ripped out stream-level sendmes, then as you say, we'd have to
choose between "queue all the data for the stream, no matter how big it
gets" and "tell the whole circuit to shut up".

I believe you have just poked a hole in the n23 ("defenstrator") design
as well: http://freehaven.net/anonbib/#pets2011-defenestrator
since it lacks any stream-level pushback for streams that are blocking
on writes. Nicely done!

--Roger