[tor-dev] Development of an HTTP PT

[vmon at riseup.net]

Hey dardok,

I really wanted to reach out to you earlier but I was very busy
delivering Stegotorus deliverables at the time.

>>>> I've read some papers [0],[1],[2],[3] and the ticket #8676 and I
>>>> consider that it would be a good idea to make an effort and try
>>>> to implement the HTTP PT as is stated in the ticket, that is
>>>> using real browser and server services.

Currently there is a steg module for Stegotorus which does that for the
server side (http_apache). On the client side, I started modularizing
the client that is actually sending the http request. So anybody can
write a client (let say a Firefox or chrome plugin) which take care of
sending the request and feed back the reply to Stegotorus. I did a bit
of work here:

https://github.com/zackw/stegotorus/tree/f--ext-cover-client-comms

But it was a bit more time-consuming than the tight deadlines I had
. So, I compromised on curl for a  client side for now. So
currently, on the client side, Stegotorus gives the URL to curl and curl
send them and give back the response to Stegotorus. curl for example, can
fake user agent but it would be great to have the user's own Firefox on
the other end. So if you think it is something that interests you we can
work on it together.

If you want to write a transport from Scratch, Kevin's point about
"initial high-level goals" is very essential. If your only concerns is
to adhere to valid HTTP 1.1 protocol then something like a "tcp over http"
obfsproxy protocol shouldn't be that hard to implement. But if you want
something that give a minimal resemblence to what usually look like 
HTTP on the Internet (and I assume you want that, otherwise why
to go so much out of your way to pretend using a real browser and a http
server) then the chopping and re-assembly will become the real
pain. 

The good thing about Stegotorus is that it is very modularized and you
can easily do something like this:

TBB <-> Scramblesuit PT <-> Stegotorus as Chopper <-> HTTP PT <-> CENSOR NET

What I was aiming was

TBB <-> Scramblesuit PT <-> Stegotorus as Chopper <-> HTTP PT <-> Modified browser <-> CENSOR NET

And now is
TBB <-> Scramblesuit PT <-> Stegotorus as Chopper <-> HTTP PT <-> curl <-> CENSOR NET

If you want to go extremely ambitious then something like Infarnet is
the answer but with that channel capacity I highly doubt if you can even get a
tor circuit bootstrapped.

Let me know if you are interested to work together on this.

Cheers,
vmon

Kevin P Dyer <kpdyer at gmail.com> writes:
> Hi dardok,
>
> Thanks for thinking about HTTP pluggable transports. It turns out to
> be a deceptively hard problem!
>
> Do you have any initial high-level goals? Or are you just trying to
> figure out *some* way to bootstrap an existing HTTP client/server
> architecture to tunnel arbitrary data streams?
>
> -Kevin
>
> On Sun, Nov 17, 2013 at 6:22 AM, dardok <dardok at riseup.net> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA512
>>
>> George Kadianakis:
>>> dardok <dardok at riseup.net> writes:
>>>
>>>> Hi, I am quite new in here but I am interested to help and
>>>> improve the TOR system. I am interested in PTs and particularly
>>>> in developing a HTTP PT.
>>>>
>>>> I've read some papers [0],[1],[2],[3] and the ticket #8676 and I
>>>> consider that it would be a good idea to make an effort and try
>>>> to implement the HTTP PT as is stated in the ticket, that is
>>>> using real browser and server services.
>>>>
>>>> After talking with asn, we conclude that a good point to start
>>>> this development may be to focus on the HTTP transport part, that
>>>> is to know how to control the browser or the server and how to
>>>> embed the TOR traffic into the HTTP protocol (requests and
>>>> responses). Things such as the data obfuscation, the delays in
>>>> the communications and the packet chopping won't be considered,
>>>> because it may be used another PT such Scramblesuit to do that
>>>> task.
>>>>
>>>> The CLIENT side:
>>>>
>>>> TBB <-> Scramblesuit PT <-> HTTP PT <-> CENSOR NET
>>>>
>>>> and the SERVER side:
>>>>
>>>> CENSOR NET <-> HTTP PT <-> Scramblesuit PT <-> TOR bridge
>>>>
>>>> The important is to know how to embed the TOR traffic already
>>>> obfuscated into the requests and responses to avoid suspicion.
>>>> Also as I said before, to know how to control a browser binary to
>>>> make the HTTP traffic from the client side as much traditional as
>>>> possible, for instance using a firefox binary or something like
>>>> that. The same must be applied to the server side, implementing a
>>>> real NGINX server or an Apache server on port 80 and writting
>>>> some CGI to classify the traffic incoming from the TOR clients
>>>> through the HTTP requests. The same server may have another CGI
>>>> to write and send the HTTP responses to those TOR clients with
>>>> the traffic into them.
>>>>
>>>> I would like to find someone interested to work on this topic.
>>>>
>>>
>>> Hey there,
>>>
>>> we discussed this project on IRC and looking at your post it seems
>>> that you understood things :)
>>>
>>> Like I told you, I'm interested in this topic, but my free time is
>>> miniscule these days. Still, I'd like to help you out. Do you know
>>> what you need help with?
>>>
>>> If you want, we can organize a meetup in IRC to discuss and plan
>>> future work on this.
>>>
>>> It would be great if you prepared a bit of research on the basic
>>> components of this project [0] so that we can discuss the various
>>> options during the meeting. I'll send you an email to find the
>>> right date and time for the meeting :)
>>>
>>> Cheers!
>>>
>>> [0]: ways to control a browser from within a PT (selenium?), HTTP
>>> covert channels, designs of how a server-side CGI script would
>>> work, etc. _______________________________________________ tor-dev
>>> mailing list tor-dev at lists.torproject.org
>>> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
>>>
>>
>> Hi,
>>
>> I've been reading about Selenium web-browser driver thing and I
>> consider that is not very handy to do what an HTTP PT client side
>> needs, that is to forge HTTP requests and embbed the TOR traffic into
>> these HTTP requests.
>>
>> It is more oriented to emulated a web user interaction with a real
>> browser (such as firefox, chrome, opera, ie, ...) and the interaction
>> and testing of web apps from these browsers.
>>
>> Also I cannot see how to handle the responses from the server-side
>> using this thing. Few functions seem to be interesting regarding HTTP
>> protocol handling and manipulation, as I said before most of the
>> functions present are related with the user interaction.
>>
>> The Python version can be installed easily: pip install selenium
>>
>> The already implemented functions can be read on this file:
>> /usr/share/pyshared/selenium/selenium.py
>>
>> I didn't find any useful function to allow a HTTP PT (client side)
>> embed the TOR traffic into HTTP requests (maybe some cookie function
>> related) and extract the information sent by the HTTP PT server side
>> (maybe some get_body_text() or get_text() funtion).
>>
>> So I am ready to consider that this option is not be useful to
>> implement a HTTP PT client side. Anyway, I would like to discuss this
>> point with someone interested in this topic.
>>
>> Thanks!
>> -----BEGIN PGP SIGNATURE-----
>>
>> iQEcBAEBCgAGBQJSiNE0AAoJEFz9RJtDk2+MecQH/31EBnd7IVgoEOQGK60bmdY9
>> JV6sRpd0JNgnV1au17GPOy+jcRUIHvL9dLWW1VJzf1wLk23wx2wb5xALBzpvwPiC
>> TC32YWZBqqtalY8u2f3j7+ScjZdps69dIP3idp7P113nEogBV05pPxw2RKf2IaCM
>> IUN9jvAsM7ITk3t+Oeg34BcW8CYJuTTyO87z7VfDiO8ZbyABC7fgpBOs5O0jWtps
>> jXyY3HeExldmVBacPNqY5HAlJ5XHC9dQn14MtjCcoTxVY6lzn51RFZUDPjJxc5D9
>> L5tGrB6dLcT/SUSU2jtHvrEN8tIQg5sGJHngfPurwO9gMhtbuKFLbKfwUB7Op+w=
>> =vIHR
>> -----END PGP SIGNATURE-----
>> _______________________________________________
>> tor-dev mailing list
>> tor-dev at lists.torproject.org
>> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
> _______________________________________________
> tor-dev mailing list
> tor-dev at lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev