[tor-dev] Proposal: Move IPv6 ORPorts to the Microdesc Consensus
teor
teor2345 at gmail.com
Tue Nov 21 13:37:52 UTC 2017
> On 2 Nov 2017, at 07:22, Nick Mathewson <nickm at torproject.org> wrote:
>
>> On Tue, Oct 24, 2017 at 11:35 PM, teor <teor2345 at gmail.com> wrote:
>> Hi all,
>>
>> We would like to move IPv6 ORPorts from microdescriptors to the
>> microdesc consensus. This makes it easier for IPv6 clients to bootstrap
>> and choose reachable guards.
>>
>> The proposal is inlined below, it is also available with the corresponding
>> dir-spec updates in my torspec branch bug23826-23828 on GitHub:
>>
>> https://github.com/teor2345/torspec.git
>>
>> The tor code that implements these new consensus methods is in my tor
>> branch on bug23826-23828 on GitHub:
>>
>> https://github.com/teor2345/tor.git
>>
>> The parent ticket for these related changes is #20916. The code changes are
>> being tracked in #23826 and #23828, and the spec changes and proposal in
>> #23898:
>>
>> https://trac.torproject.org/projects/tor/ticket/20916
>>
>> If we've spoken about this, and I've left you out as an author, please let
>> me know!
> Hi, Tim! I promised you a quick review here, so here goes. I have some questions, but nothing looks like a showstopper here.
Hi Nick, sorry it's taken me so long to reply - some more urgent tasks came up!
I've revised the proposal in my bug23826-23828-v2 branch on GitHub:
https://github.com/teor2345/torspec/blob/bug23826-23828-v2/proposals/283-ipv6-in-micro-consensus.txt
>> Here is the proposal text:
>>
>> Filename: xxx-ipv6-in-micro-consensus.txt
>> Title: Move IPv6 ORPorts from microdescriptors to the microdesc consensus
>> Author: Tim Wilson-Brown (teor)
>> Created: 18-Oct-2017
>> Status: Open
>> Target: 0.3.3.x
>>
>> 1. Summary
>>
>> Moving IPv6 ORPorts from microdescs to the microdesc consensus will make
>> it easier for IPv6 clients to bootstrap and select reachable guards.
>>
>> Since consensus method 14, authorities have voted for IPv6 address/port
>> pairs (ORPorts) in "a" lines. Unreachable IPv6 ORPorts are dropped from the
>> full consensus. But for clients that use microdescriptors (the default),
>> IPv6 ORPorts are placed in microdescriptors. So these clients can only tell
>> if an IPv6 ORPort is unreachable when a majority of voting authorities
>> mark the relay as not Running.
>>
>> This proposal puts reachable relay IPv6 ORPorts in an "a" line in the
>> microdesc consensus. This allows clients to discover unreachable IPv6
>> ORPorts, even if a minority of voting authorities set
>> AuthDirHasIPv6Connectivity 1.
>>
> To me, this motivation makes a little less sense than the bootstrapping improvements in 4.3 do.
I added the bootstrapping improvements to the summary, and edited the
unreachable bit to make it shorter.
> Don't get me wrong: it's cool that we can get IPv6 online-ness detection "for free" on existing clients... but there are other ways we could IPv6 online-status advertising too (like a status flag,
Yes, like NoEdConsensus.
> or conditionally omitting "a" lines from microdescriptors,
No, we can't do that: microdescriptors are a deterministic transform from
descriptors. They only depend on the consensus method.
> or something else).
>
> But the bootstrapping considerations discussed in 4.3 below are something we really _can't_ do without moving the "a" lines into the consensus. So to my mind, that's the major reason we should do this.
Let me know if the new summary makes sense.
>> 2. Proposal
>>
>> We add two new consensus methods, here represented as M and N (M < N), to
>> be allocated when this proposal's implementation is merged. These consensus
>> methods move IPv6 ORPorts from microdescs to the microdesc consensus.
>>
>> We use two different methods because this allows us to modify client code
>> based on each method. Also, if a bug is discovered in one of the methods,
>> authorities can be patched to stop voting for it, and then we can implement
>> a fix in a later method.
>>
>> 2.1. Add Reachable IPv6 ORPorts to the Microdesc Consensus
>>
>> We specify that microdescriptor consensuses created with methods M or later
>> contain reachable IPv6 ORPorts.
>>
>> 2.2. Remove IPv6 ORPorts from Microdescriptors
>>
>> We specify that microdescriptors created with methods N or later do not
>> contain any IPv6 ORPorts.
> Let's say that with method N, we start omitting them. Let's not say that we commit to omitting them forever. Perhaps we will someday have a reason to put more "a" lines in microdescriptors again.
Done!
>
>> 3. Retaining Existing Behaviour
>>
>> The following existing behaviour will be retained:
>>
>> 3.1. Authority IPv6 Reachability
>>
>> Only authorities configured with AuthDirHasIPv6Connectivity 1 will test
>> IPv6 ORPort reachability, and vote for IPv6 ORPorts.
>>
>> This means that:
>> * if no voting authorities set AuthDirHasIPv6Connectivity 1, there will be
>> no IPv6 ORPorts in the consensus,
>> * if a minority of voting authorities set AuthDirHasIPv6Connectivity 1,
>> unreachable IPv6 ORPort lines will be dropped from the consensus, but the
>> relay will still be listed as Running,
>> * if a majority of voting authorities set AuthDirHasIPv6Connectivity 1,
>> relays with unreachable IPv6 ORPorts will be dropped from the consensus.
>>
>> We will document this behaviour in the tor manual page, see #23870.
> So, there's an alternative here: we could let the HasIPV6 authorities vote on a flag to indicate "reachable/unreachable with IPv6," and let all the authorities vote on the "a" lines. Then, in the consensus, we could omit the "a" lines unless the router has the reachable-with-ipv6 flag; and include them otherwise.
>
> This way, we wouldn't need to have a majority of IPv6 authorities in order to have meaningful "a" lines that tell you whether the router is reachable. (But of course, the more we had, the more reliable the information would be.)
>
> This change could be done as part of consensus method M, I think. Do you think it's worthwhile?
We already have meaningful "a" lines in the full consensus, because authorities only
vote on them when they are configured with AuthDirHasIPv6Connectivity 1 and when they
find the relay reachable. Otherwise, they don't vote on "a" lines.
And then when compute_routerstatus_consensus() tallies the votes, it ignores votes
without "a" lines, and chooses the most popular IPv6 address in the votes for the most
popular descriptor.
I have updated the proposal and man page patch in #23826 to make this clearer.
Here's another alternative we discarded:
We could remove the IPv6 addresses from relays with unreachable IPv6 ORPorts, but keep
their IPv4 addresses. I asked the directory authority operators, and their feedback was
that it's better to mark a relay "not Running" if any of its ORPorts are unreachable or
misconfigured.
>> 3.2. Full Consensus IPv6 ORPorts
>>
>> The full consensus will continue to contain reachable IPv6 ORPorts.
> By "full" consensus, do you mean "NS" consensus? I don't think we use "full" elsewhere.
Oops.
I'll replace "full consensus" with "NS consensus" in the proposal.
>> 3.3. Clients that use Full Descriptors
>>
>> Tor clients that use full descriptors already ignore unreachable IPv6
>> ORPorts, and have done so since at least 0.2.8.x.
> Wow. I'd forgotten this. How does this work?
Oh, apparently it doesn't.
(More precisely, it only works for IPv6-enabled authorities, because they have
their own non-consensus opinions about IPv6 addresses.)
I've deleted this section, and made minor edits to the rest of the proposal to
clarify.
>> 4. Impact and Related Changes
>>
>> 4.1. Directory Authority Configuration
>>
>> We will work to get a super-majority (75%) of authorities checking relay
>> IPv6 reachability, to avoid Running-flag flapping. To do this, authorities
>> need to get IPv6 connectivity, and set AuthDirHasIPv6Connectivity 1.
> How far away are we from this today?
We have 5/9, and need 7/9, because the remaining 2 authorities will be a minority
in a 5-vote consensus. Realistically, we don't often get below 7, so 6/9 would
be sufficient for most consensuses.
Running-flag flapping is annoying, but it isn't too bad: if relays are only
included for a few consensuses, they won't get much bandwidth.
> How long do the authority operators think it would take?
Good question!
I emailed the dir-auth list about IPv6 about a month ago, and since 23 October,
5/9 authorities have IPv6 available and configured.
For the remainder, you'll need to ask Roger, Micah, Sina, or Alex.
But I think they're moving as fast as they can.
Micah and Roger have had IPv6 before, but they don't have working IPv6 in their
networks at the moment.
Alternately, we could add a 10th authority with IPv6, which would give us 6/10,
which would be sufficient for most consensuses. And 7/10 would be a
super-majority.
>> 4.2. Relays and Bridges
>>
>> Tor relays and bridges do not currently use IPv6 ORPorts from the
>> consensus.
>>
>> We expect that 2/3 of authorities will be voting for consensus method N
>> before future Tor relay or bridge versions use IPv6 ORPorts from the
>> consensus.
>>
>> 4.3. Clients
>>
>> 4.3.1. Legacy Clients
>>
>> 4.3.1.1. IPv6 ORPort Circuits
>>
>> Tor clients on versions 0.2.8.x to 0.3.2.x check directory documents for
>> ORPorts in the following order:
>> * descriptors (routerinfo, available if using bridges or full descriptors)
>> * consensus (routerstatus)
>> * microdescriptors (IPv6 ORPorts only)
>>
>> Their behaviour will be identical to the current behaviour for consensus
>> methods M and earlier. When consensus method N is used, they will ignore
>> unreachable IPv6 ORPorts without any code changes.
>>
>> 4.3.1.2. IPv6 ORPort Bootstrap
>>
>> Tor clients on versions 0.2.8.x and 0.2.9.x are currently unable to
>> bootstrap over IPv6 only connections when using microdescriptors. This
>> happens because the microdesc consensus does not contain IPv6 ORPorts.
>>
>> When consensus method M is used, they will be able to bootstrap over IPv6
>> only connections using microdescriptors, without any code changes.
> (How does the behavior of 0.3.0.x and onward differ here?)
Added a clarifying sentence:
IPv6-only Tor clients on versions 0.3.0.2-alpha and later use fallback
directory mirrors to fetch their microdescriptors.
>> 4.3.2. Future Clients
>>
>> 4.3.2.1. Ignoring IPv6 ORPorts in Microdescs
>>
>> Tor clients on versions 0.3.3.x and later will ignore unreachable IPv6
>> ORPorts once consensus method M or later is in use. (See #23827.)
>>
>> 4.3.2.2. IPv6 ORPort Bootstrap
>>
>> If a bootstrapping IPv6-only client has a consensus made with method M or
>> later, it should download microdescriptors from one of the IPv6 ORPorts in
>> that consensus. Previously, IPv6-only clients would use fallback directory
>> mirrors to download microdescs, because there were no IPv6 ORPorts in the
>> microdesc consensus. (See #23827.)
>>
>> 4.3.2.3. Ignoring Addresses in Unused Directory Documents
>>
>> If a client doesn't use a particular directory document type for a node,
>> it should ignore any addresses in that document type. (See #23975.)
>>
>> 5. Data Size
>>
>> This change removes 2-50 bytes from the microdescriptors of relays that
>> have an IPv6 ORPort, and adds them to reachable IPv6 relays' microdesc
>> consensus entries.
>>
>> As of October 2017, 600 relays (9%) have IPv6 ORPorts in the full
>> consensus. Their "a" lines take up 19 KB, or 33 bytes each on average.
>> The microdesc consensus is 1981 KB, so this represents about 1% of its
>> uncompressed size.
>>
>> Most tor clients are already running 0.3.1.7, which implements consensus
>> diffs. We expect that most directory mirrors will also implement consensus
>> diffs by the time 2/3 of authorities are voting for consensus method M.
>>
>> So we expect that this change will have a minimal impact, which is made
>> even smaller by compression and consensus diffs.
> Let's look at a worst-case analysis, though. How would the impact be if 100% of the relays had IPv6 ORPorts? I'm not very interested in the uncompressed size; it's the gzip-compressed size that determines the worst-case impact.
In the worst-case, the gzip-compressed size is 2% larger with the current
IPv6 ORPorts, and 14% larger with 100% IPv6 ORPorts.
The zstd-compressed size is 1.4% larger with the current IPv6 ORPorts, and
8% larger with 100% IPv6 ORPorts. But the zstd consensus is actually larger
than the gzip one:
https://trac.torproject.org/projects/tor/ticket/24368
>> 6. External Impacts
>>
>> We don't expect this change to impact Onionoo and similar projects, because
>> they typically use the full consensus.
>>
>> Metrics doesn't currently graph IPv6 usage in Tor, but would like to in
>> future.
I've updated this section, and added a "Monitoring" section.
T
--
Tim Wilson-Brown (teor)
teor2345 at gmail dot com
PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B
ricochet:ekmygaiu4rzgsk6n
xmpp: teor at torproject dot org
------------------------------------------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20171122/ef8744fc/attachment.sig>
More information about the tor-dev
mailing list