[tor-dev] New Proposal 306: A Tor Implementation of IPv6 Happy Eyeballs

[Iain Learmonth]

Hi,

My comments are inline.

> Filename: 306-ipv6-happy-eyeballs.txt Title: A Tor Implementation of
> IPv6 Happy Eyeballs Author: Neel Chauhan Created: 25-Jun-2019 
> Supercedes: 299 Status: Open Ticket:
> https://trac.torproject.org/projects/tor/ticket/29801
> 
> 1. Introduction
> 
> As IPv4 address space becomes scarce, ISPs and organizations will
deploy
> IPv6 in their networks. Right now, Tor clients connect to guards
> using IPv4 connectivity by default.
> 
> When networks first transition to IPv6, both IPv4 and IPv6 will be
enabled
> on most networks in a so-called "dual-stack" configuration. This is
to not
> break existing IPv4-only applications while enabling IPv6
> connectivity. However, IPv6 connectivity may be unreliable and
> clients should be able to connect to the guard using the most
> reliable technology, whether
IPv4
> or IPv6.

The big problem that happy eyeballs was meant to solve was that often
you might have something announcing an IPv6 prefix but that routing was
not properly configured, so while the operating system thought it had
IPv6 Internet it was actually just broken. In some cases, the IPv6
Internet would be partitioned as there weren't enough backup routes to
fail over to in times of outages. For most purposes, as I understand it,
this means either IPv6 connectivity to a host is there or it's not.
There's not really a middle ground where it sometimes works but is flaky
(i.e. where you can maintain a connection but it has high packet loss).

> In ticket #27490, we introduced the option ClientAutoIPv6ORPort
> which lets a client randomly choose between IPv4 or IPv6. However,
> this random decision does not take into account unreliable
> connectivity or falling back to the competing IP version should one
> be unreliable or unavailable.
> 
> One way to select between IPv4 and IPv6 on a dual-stack network is a 
> so-called "Happy Eyeballs" algorithm as per RFC 8305. In one, a
> client attempts the preferred IP family, whether IPv4 or IPv6. Should
> it work, the client sticks with the preferred IP family. Otherwise,
> the client attempts the alternate version. This means if a dual-stack
> client has both IPv4 and IPv6, and IPv6 is unreliable, preferred or
> not, the client uses IPv4, and vice versa. However, if IPv4 and IPv6
> are both equally reliable, and IPv6 is preferred, we use IPv6.

This sounds like a good candidate for a consensus parameter, such that
we can switch the preference for all clients at once, not just the ones
that have updated to the version we switch the preference in.

There may also be other ordering parameters for the address candidates.
We might want to avoid using IPv6 addresses that are using 6to4 or
Teredo as we *know* those are tunnels and thus have encapsulation
overhead, higher latency, and funnel all the traffic through centralised
(even if distributed) points in the network.

> In Proposal 299, we have attempted a IP fallback mechanism using
failure
> counters and preferring IPv4 and IPv6 based on the state of the
counters.
> However, Prop299 was not standard Happy Eyeballs and an alternative, 
> standards-compliant proposal was requested in [P299-TRAC] to avoid
issues
> from complexity caused by randomness.
> 
> This proposal describes a Tor implementation of Happy Eyeballs and
> is intended as a successor to Proposal 299.
> 
> 2. Address Selection
> 
> To be able to handle Happy Eyeballs in Tor, we will need to modify
> the data structures used for connections to guards, namely the extend
> info structure.
> 
> The extend info structure should contain both an IPv4 and an IPv6
address.
> This will allow us to try IPv4 and the IPv6 addresses should both be 
> available on a relay and the client is dual-stack.

The Happy Eyeballs specification doesn't just talk about having one v4
and one v6 address. In some cases, relays may be multihomed and so may
have multiple v4 or v6 addresses. We should be able to race all the
candidates.

> When parsing relay descriptors and filling in the extend info data 
> structure, we need to fill in both the IPv4 and IPv6 address if
they both
> are available. If only one family is available for a relay (IPv4 or
IPv6),
> we should fill in the address for preferred family and leave the
alternate
> family null.

To match the IETF protocol more closely, we should have a list of
candidate addresses and order them according to our preferences.

> 3. Connecting To A Relay
> 
> If there is an existing authenticated connection, we should use it 
> similar to how we used it pre-Prop306.
> 
> If there is no existing authenticated connection for an extend info, 
> we should attempt to connect using the first available, allowed, and 
> preferred address.
> 
> We should also allow falling back to the alternate address. For
> this, three alternate designs will be given.
> 
> 3.1. Proposed Designs
> 
> This subsection will have three proposed designs for connecting to
relays
> via IPv4 and IPv6 in a Tor implementation of Happy Eyeballs. The
proposed
> designs are as listed as follows:
> 
> * Section 3.1.1: First Successful Authentication
> 
> * Section 3.1.2: TCP Connection to Preferred Address On First
Authenticated
> Connection
> 
> * Section 3.1.3: TCP Connection to Preferred Address On First TCP
Success
> 
> 3.1.1. First Successful Authentication
> 
> In this design, Tor will first connect to the preferred address and 
> attempt to authenticate. After a 1.5 second delay, Tor will connect 
> to the alternate address and try to authenticate. On the first 
> successful authenticated connection, we close the other connection.
> 
> This design places the least connection load on the network, but 
> might add extra TLS load.

The delay seems arbitrary. OnionPerf collects data on latency in the Tor
network, and could be used to inform better timing choices for the best
end user performance (the happiest eyeballs).

If we choose to take this route, we should open new connections with a
timeout of ~250ms, and only change the condition for deciding which is
the connection we will use.

> 3.1.2. TCP Connection to Preferred Address On First Authenticated
Connection
> 
> This design attempts a TCP connection to a preferred address. On a 
> failure or a 250 ms delay, we try the alternative address.
> 
> On the first successful TCP connection Tor attempts to authenticate 
> immediately. On the authentication failure, or a 1.5 second delay, 
> Tor closes the other connection.
> 
> This design is the most reliable for clients, but increases the 
> connection load on dual-stack guards and authorities.

Creating TCP connections is not a huge issue, and we should be racing
the connections with the ~250ms timeout anyway. All the designs will
have this issue.

> 3.1.3. TCP Connection to Preferred Address On First TCP Success
> 
> In this design, we will connect via TCP to the first preferred
> address. On a failure or after a 250 ms delay, we attempt to connect
> via TCP to the alternate address. On a success, Tor attempts to
> authenticate and closes the other connection.
> 
> This design is the closest to RFC 8305 and is similar to how Happy 
> Eyeballs is implemented in a web browser.

This is probably also the "simplest" to implement, as it means that the
happy eyeballs algorithm is contained to the socket handling code.

I don't believe that requiring authentication to complete is going to do
anything more than generate load on relays. Either the packet loss is
high enough that the three way handshake fails, or there is low packet
loss. I don't think the case where requiring an additional few packets
make it through helps you choose a better connection is going to be that
common.

Of course it is always possible to add a "PreferredAddressFamily" option
to torrc for those that know they are on a bad IPv6 network.

> 3.2. Recommendations for Implementation of Section 3.1 Proposals
> 
> We should start with implementing and testing the implementation as 
> described in Section 3.1.1 (First Successful Authentication), and
> then doing the same for the implementations described in 3.1.2 and
> 3.1.3 if desired or required.

I'd want to see some justification with some experimental (or even
anecdotal) data as to why first successful authentication is the way to
go. 3.1.3 is going to be the simpler option and, in my opinion, the best
place to start.

3.1.3 can likely be implemented using exactly the algorithm in section 5
of RFC 8305, excluding portions relating to DNS because we already have
all the candidates from the server descriptor.

> 4. Handling Connection Successes And Failures
> 
> Should a connection to a guard succeed and is authenticated via TLS,
> we can then use the connection. In this case, we should cancel all
> other connection timers and in-progress connections. Cancelling the
> timers is so we don't attempt new unnecessary connections when our
> existing connection is successful, preventing denial-of-service
> risks.
> 
> However, if we fail all available and allowed connections, we
should tell
> the rest of Tor that the connection has failed. This is so we can
attempt
> another guard relay.

Some issues that come to mind:

- I wonder how many relay IPv6 addresses are actually using tunnels. At
the levels of throughput they use, that overhead adds up. What is the
additional bandwidth cost and what is the impact of reduced MSS?
- What are the tunables? RFC8305 has some that would be applicable, and
probably all of them could be consensus parameters if we wanted to tune
them:
* First Address Family Count
* Connection Attempt Delay
* Minimum Connection Attempt Delay
* Maximum Connection Attempt Delay
- How do we know what is going on? We do not collect metrics from
clients about their usage, but we do collect metrics from relays. Are
there any counters we should be adding to extra info descriptors to help
us see whether or not this is working? Could clients help relays by
reporting that a connection is being closed because they have another
connection? (I don't know the answer, but RFC8305 does explicitly point
out that it is a mitigation technique designed to hide problems from the
user, which means that those problems might come back to haunt us later
if we're not on top of them.)

Thanks,
Iain.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20190702/fc7a4935/attachment.sig>