[tor-dev] [Stegotorus] Fundamental problem with ack/retransmission mechanism

[Vmon]

Hey Zack,

I'm not sure if you were following Iranian filtering few days leading to
the election. It was basically http white-list. Psipohn was sending few 'GET
/ HTTP 1.1' before start sending any real data and it was able to fool
the box . But the filtering is going to get more intelligent next time,
and hence I feel stegotorus is more important than ever. I couldn't test
stegotorus, cause SSH wasn't working and I couldn't access my vps there :(

My time freed up a bit and I thought we should deal with the ack design
problem once and forever. 

The fundamental thing that is not clear for me is that "If we have the
assumption that TCP is reliable, this means all packets are going to
arrive sooner or later. In such situation what's the need for
re-transmission".

The only scenario I can imagine where retransmit is useful is the following:
Re-assembly queue has 255 rooms. If a packet is delayed so much that
misses 255 packets after it then we are not able to reconstruct the
information. Otherwise if our re-assembly queue had infinite length, at
least theoretically we didn't need to have ack/retransmission at all
cause TCP reassurance was enough for us. This situation is imaginable
only if we have more than one connection otherwise TCP would assure also
the order in which our pseudo-packets are arriving.

Is that statement correct? If it is correct, then we can modify the
design to deal with above situation instead of being a full-fledged ack
mechanism. That will save lots of traffic instead of blindly
retransmitting big packets all the time. 

Thanks for helping me with this.

Best,
Vmon

Zack Weinberg <zackw at panix.com> writes:

> On Thu, Apr 18, 2013 at 5:08 PM,  <vmonmoonshine at gmail.com> wrote:
>> Zack Weinberg <zackw at panix.com> writes:
>>>and the retransmit of packet 1 ought to be happening on a
>>>*different* connection, if we have one.  (Which steg are you using?)
>> I'm using nosteg steg. I thought if something is going to work, better, it
>> works with the simplest steg. nosteg only open one connection and pass
>> everything through it as long as there is no reason to drop it, for bad
>> header etc.
>
> OK. We shouldn't even try to retransmit with a 1-connection steg mode
> (unless it's not using TCP ... worry about that later)
>
>>> So I'm not understanding exactly what the "dropper proxy" does.  Does
>>> it prevent TCP from providing reliable delivery?  If so, how?
>> It sits at socket_read_cb and sometimes doesn't copy what libevent's has
>> read from one side of communication, into the buffer of the other
>> side.
>
> Ah.  Yeah, that's not going to work.  TCP will think that the data
> _has_ been delivered, so the lower-level retransmit that we're relying
> on will never happen.
>
>> So, what you say, means that libevents socket_read_cb calls are more
>> refined than an entire TCP packet that TCP guaranteed to deliver. I.e,
>> libevent break something that TCP guaranteed to deliver in smaller
>> parts.
>
> TCP is a stream-oriented protocol.  It guarantees to provide reliable,
> ordered delivery of a _sequence of bytes_.  It does _not_ guarantee
> anything whatsoever about packet boundaries.  In particular, the
> amount of data libevent hands you in one read callback is completely
> meaningless.
>
>> If this is true, I guess because TCP is giving us a stream there is no
>> way for us to know where to drop to have a legitimate simulation of real
>> life packet drop in that stream, unless I incorporate part of chop in
>> the dropper proxy to read their headers and detect end of packet.
>
> What you need to do is implement the dropping _below_ TCP, so that TCP
> is aware of it and does do its retransmits.  You can do this on Linux
> with netem
> <http://www.linuxfoundation.org/collaborate/workgroups/networking/netem>
> and on many of the *BSDs (including OSX) with dummynet
> <http://info.iet.unipi.it/~luigi/dummynet/>.
>
>>> ST absolutely *does* need a congestion
>>> control mechanism, though, to prevent the entire circuit from getting
>>> killed because it overran the fixed-size reassembly queue, and as long
>>> as we have to do that ...
>>>
>> Then maybe I just start an axe timer whenever when I want to send and
>> the transmit queue is full and delete the timer whenever the queue
>> moves.
>
> How do you know that your peer's receive queue is full, if it doesn't tell you?
>
> But we probably could get away with something much simpler if we don't
> bother doing retransmits, e.g. something akin to Tor's SENDME cells.
>
> zw