[tor-dev] Channel incoming queue

[Ian Goldberg]

On Thu, May 02, 2013 at 10:58:54AM -0400, MF Nowlan wrote:
> I am working on integrating uTCP and uTLS (
> http://arxiv.org/abs/1103.0463) into Tor to see if we can lower the
> latency due to head of line blocking across circuits.

You have to be careful to preserve cell ordering *within* circuits, of
course.

[Do you know about TCP-over-DTLS and PCTCP?]

> 2. Now, look for case when two cells from different circuits are
> present in the incoming_queue and process the second cell before the
> first cell.
> channel_process_cells(...) {
> ...
>   while (NULL != (q = TOR_SIMPLEQ_FIRST(&chan->incoming_queue))) {
>     // Remove q from the incoming_queue.
>     // if the queue is still not empty, get the next one,
>     // if the circuit ids do not match, Swap the cells.
>   }
> ...
> }
> 
> My problem is that number 2. above never occurs. I have not observed
> a case where the incoming_queue ever has two cells from different
> circuits. In fact, I don't think I've ever had a time when the
> incoming_queue has more than 1 cell in it.  I am hammering a small
> tor test network with 30+ curl requests at once. I have two proxies,
> each of them uses the same entry node, and the same exit node, and
> there is only one other node in the network, so the circuit that is
> set up should be the same for every single request.  Am I missing
> something? Will this function (channel_process_cells) ever process
> more than one cell at a time? I've checked the logs to verify that
> different circuits are actually set up for the different requests
> (rather than the Proxies just reusing the existing circuit and
> giving each new request a new stream id).

I'm not very familiar with the new channel code, unfortunately (it's on
our "todo" list to port TCP-over-DTLS and/or PCTCP to channels, though).
But at least in the pre-channel code, the input queues would indeed
almost always be empty.  Any why wouldn't they be?  As soon as TLS
reports a cell has arrived, you should be able to process it
immediately.  Where the HOL blocking occurs is in the Tor *output* queue
on the sending side and/or the TCP receive buffer on the receiving side.
If a IP packet containing (...TCP...TLS...) a cell gets dropped, all
future data on that connection will get stalled in the TCP receive
buffer (if the window is open) or the Tor output queue (if the window is
closed).  Once it gets delivered to Tor, then the stall has already been
resolved.

   - Ian