[tor-dev] Linux kernel transproxy packet leak (w/ repro case + workaround)

[Mike Perry]

Hello all,

I've discovered that the Linux kernel appears to have a leak in how it
applies transproxy rules to the TCP CLOSE_WAIT shutdown condition under
certain circumstances. This applies to both the kernels in use by common
Android devices (Cyanogenmod 10.x and 11-M4), as well as the Linux
kernel in Ubuntu 13.04 (3.8.0-35-generic).

The bug can be triggered either by a remote server closing a connection,
or by restarting the local tor client.

Basically, the bug happens when a transproxy connection shuts down
completely before a client application properly closes the socket. This
seems to cause the kernel to lose track of the fact that the client
application connection was being transproxied, and when the client
application finally does close its socket (or exits), the Linux kernel
generates a FIN ACK that completely bypasses any transproxy rules you
have installed. It sends this packet first as the UID of the app in
question, and if that fails, it resends it as a blank UID (the kernel
itself).


Here's how to reproduce it and see for yourself:

First run the attached iptables script, and launch a tor daemon with the
attached torrc (edit the iptables script's TOR_UID=`id -u debian-tor`
and NETWORK_USER_ID=1000 vars if your setup is different).

Then, fire up tcpdump, like so:
 sudo tcpdump -n -i wlan0 host 74.125.28.104 and tcp port 80

Replace '-i wlan0' with your network interface. If you use '-i any', you
will also see transproxied packets (which are not normally leaked).

Then, as your transproxied user, paste this python snippet into a python
interpreter:
 import socket
 s = socket.create_connection(("74.125.28.104", 80))

(That IP handles www.google.com).

After the connection is made, you should see something like the
following in 'netstat -natp':
 tcp  0 0  127.0.0.1:9040      192.168.1.23:42235 ESTABLISHED 1121/tor        
 tcp  0 0  192.168.1.23:42235  74.125.28.104:80   ESTABLISHED 977/python      

At this point, either wait a couple minutes for Google to close that
connection on you, or shut down your Tor daemon. In either case, you
should see the first connection transition to TIME_WAIT, or FIN_WAIT2,
or similar, lose track of its PID+UID, and then finally disappear
entirely, but the python program will remain in CLOSE_WAIT indefinitely.

Once the first connection is fully gone (this takes 60s from TIME_WAIT
state with default TCP settings), issue this in your python shell:
 s.close()

At this point, you will see a FIN ACK or RST ACK packet appear in your
tcpdump window. That packet has leaked past the iptables firewall rules,
and past the transproxy rules. It went straight to Google.

I have noticed several Android apps (including Firefox, F-Droid, and
many Google apps and Android services) that allow their sockets to sit
in CLOSE_WAIT upon remote close while transproxied, and they all leak
packets in this case, which happens frequently in normal usage. I am not
sure if this is just a common programming error, an issue with how the
Android networking APIs are designed, something specifically exacerbated
by the transproxy, or some combination of these.


For a workaround, I was able to prevent this issue with the addition
of the following rules:
 iptables -I OUTPUT ! -o lo ! -d 127.0.0.1 ! -s 127.0.0.1 -p tcp -m tcp --tcp-flags ACK,FIN ACK,FIN -j DROP
 iptables -I OUTPUT ! -o lo ! -d 127.0.0.1 ! -s 127.0.0.1 -p tcp -m tcp --tcp-flags ACK,RST ACK,RST -j DROP

None of the transproxy documentation I could find mentions this issue,
nor suggests any additional safety rules. This means every transproxied
Tor user is unwittingly leaking packets, at least some of the time.
Sorry to be the bearer of bad news.

Please send workaround discussion to tor-talk, and kernel/TCP state
machine discussion to tor-dev. I Cc'd both like a jerk, because I figure
each group might have different sets of commentary, and both groups
should be aware of this issue. Don't be a jerk like me, please. Use your
best judgment to Cc one list or the other.



-- 
Mike Perry
-------------- next part --------------
#!/bin/bash

IPTABLES=/sbin/iptables
TOR_UID=`id -u debian-tor`
NETWORK_USER_ID=1000

# Clear existing rules
$IPTABLES -F INPUT
$IPTABLES -F OUTPUT 
$IPTABLES -t nat -F

## Transproxy rules for Tor
$IPTABLES -t nat -A OUTPUT ! -d 127.0.0.1 -m owner ! --uid-owner $TOR_UID -p tcp -j REDIRECT --to-ports 9040 || exit
$IPTABLES -t nat -A OUTPUT -p udp -m owner ! --uid-owner $TOR_UID -m udp --dport 53 -j REDIRECT --to-ports 5300 || exit

# Allow Tor and the network user
$IPTABLES -A OUTPUT -m owner --uid-owner $TOR_UID -j ACCEPT || exit
$IPTABLES -A OUTPUT -m owner --uid-owner $NETWORK_USER_ID -j ACCEPT
$IPTABLES -A INPUT -j LOG --log-prefix "OUTPUT DROPPED: " --log-uid || exit
$IPTABLES -A OUTPUT -j DROP || exit


# Create INPUT firewall. Allow established connections and transproxy
$IPTABLES -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT || exit
$IPTABLES -A INPUT -i lo -j ACCEPT # Transproxy output comes from lo
$IPTABLES -A INPUT -d 127.0.0.1 -m udp -p udp --dport 5300 -j ACCEPT || exit
$IPTABLES -A INPUT -j LOG --log-prefix "INPUT DROPPED: " --log-uid || exit
$IPTABLES -A INPUT -j DROP || exit
-------------- next part --------------
RunAsDaemon 1
DataDirectory /var/lib/tor
Log info file /var/lib/tor/log
User debian-tor
DNSPort 5300
TransPort 9040
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Digital signature
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20140328/05230da5/attachment.sig>