[tor-dev] prop224: What should we do with torrc options?

Mon Dec 5 23:09:57 UTC 2016

> On 6 Dec. 2016, at 09:56, David Goulet <dgoulet at ev0ke.net> wrote:
> 
> On 06 Dec (09:43:20), teor wrote:
>> 
>>> On 6 Dec. 2016, at 08:02, David Goulet <dgoulet at ev0ke.net> wrote:
>>> 
>>> On 06 Dec (07:53:16), teor wrote:
>>>> 
>>>>> On 6 Dec. 2016, at 07:42, David Goulet <dgoulet at ev0ke.net> wrote:
>>>>> 
>>>>> On 22 Nov (17:36:33), David Goulet wrote:
>>>>>> Hi everyone!
>>>>>> 
>>>>>> We are soon at the stage of implementing the service part of proposal 224
>>>>>> (next gen. hidden service.). Parent ticket is #20657 but I'll be discussing
>>>>>> here ticket #18054.
>>>>>> 
>>>>>> In a nutshell, we need to figure out the interface for the torrc file[1]. We
>>>>>> currently have some options to configure an hidden service and the question is
>>>>>> now how do we proceed on using those for next version?
>>>>> 
>>>>> [snip]
>>>>> 
>>>>> (Please read original email for some initial context.)
>>>>> 
>>>>> So over the last week, we've mashed up all the things that were said in this
>>>>> thread, me and asn discussed it with some arma also on the side!
>>>>> 
>>>>> I think the following is the best of all the non ideal solutions we can come
>>>>> up with. Below is a superb ascii timeline of what we plan to do in terms of
>>>>> transition from v2 to v3:
>>>>> 
>>>>> 
>>>>> Our dystopian reality of now.
>>>>> (https://youtu.be/NrmMk1Myrxc
>>>>> it's not a Black Mirror SO3 trailer)
>>>>>      v
>>>>>      |             ~Aug '17          ~Dec '17        Unknown date
>>>>>      |-----------------|-----------------|----------------|---------->
>>>>>                        |                 ^                ^
>>>>>                        ^            v3 Network/Code       |
>>>>>               tor stable release       Maturity           |
>>>>>                 with v3 support                       No more v2
>>>>>                    (0.3.1)
>>>>> 
>>>>> Ok, might not be my best ascii art work but I hope it will do. In short:
>>>>> 
>>>>> - With 0.3.1 (scheduled for August 2017, subject to change), we'll have a tor
>>>>> with v3 support BUT v2 will still be the default value for *new* HS.
>>>>> (HiddenServiceVersion option)
>>>>> 
>>>>> - For a period of ~4 months we estimate, we'll hope that enough of the network
>>>>> has upgraded to support v3 (relay and HSDir support are in 030) and that the
>>>>> code as some sort of maturity that we are confident to switch and make all
>>>>> new HS be v3. This is the "v3 Network/Code Maturity" marker. Note that it
>>>>> could easily go to 2018 for this switch to happen.
>>>>> 
>>>>> - When the switch happens, there will be, most likely years, a long period of
>>>>> time where v2 will be deprecated and warnings given to users but still used.
>>>>> That "Unknown date" will be the time we will release a tor stable version
>>>>> _without_ v2 support at all. It's quite unknown when we'll be able to do it.
>>>>> Chances are that we'll rely on our HS statistics and metrics for that.
>>>>> 
>>>>> That being said, here are the conclusions based on this thread and f2f
>>>>> discussion considering the above "procedure":
>>>>> 
>>>>> 1) When v3 is released in a tor stable version, it will NOT be the default
>>>>> version for new service, v2 will until maturity.
>>>>> 
>>>>> 2) HiddenServiceVersion is used to control which version of the service you
>>>>> want. Starting from the first time v3 is supported, you'll be able to use
>>>>> it but without any guarantee as we'll be entering the "stabilizing period"
>>>>> but it should be usable.
>>>>> 
>>>>> 3) ADD_ONION "BEST" will map to whatever default value HiddenServiceVersion is
>>>>> with the tor you have.
>>>>> 
>>>>> 4) In order to avoid relying on a tor stable version release to switch the
>>>>> default version from 2 to 3, we'll use a consensus parameter. Meaning that
>>>>> once we set that param. to v3, HiddenServiceVersion default value will be
>>>>> that value unless explicitely defined in torrc. It will also allow us to
>>>>> rollback to v2 if we see a swarm of badness occuring. To be clear, service
>>>>> already v3 will stay v3 but the default version for creation will simply
>>>>> change.
>>>>> ...
>>>> 
>>>> This is problematic: we validate and create onion services at torrc
>>>> load, before loading the consensus.
>>>> 
>>>> Do we plan to do it afterwards?
>>>> That seems like it could be really confusing.
>>> 
>>> You need a consensus anyway to pick your intro point or HSDir but yes the
>>> engineering part here will be that if you ever configured v2 service and you
>>> realize it's v3, we'll rollback all the things.
>>> 
>>> I agree with you that it could be confusing and will require some fun
>>> engineering but I think the advantages of having such a thing outweight the
>>> engineering effort. Unfun part will be on the user side as they will think
>>> their service is ready to start but then will have a warning after consensus
>>> download that it's not.
>>> 
>>> It also means we need to do key creation _after_ consensus download so we
>>> don't expose the wrong address to the user.
>>> 
>>> Fun... Maybe we can do better, not sure :S
>> 
>> I'd much prefer a #define for the default, and we can change it in the
>> next release.
> 
> Two downside of this. First, we make the v3 switch in a tor stable release
> which is 6 months period at the very least extending the time of adoption.
> Which also as a downside because of Linux distros updating stable package.
> Second, we lose the ability to rollback.
> 
> I guess I could be convinced that it's ok but tbh I _really_ like that we can
> decide when we want to make the switch instead of relying on stable release.
> 
>> 
>> That way, we ensure the code gets proper testing in an alpha series.
> 
> Right, I guess that's an argument against the rollback ability. I don't forsee
> us to rollback ever tbh. But also it is the whole idea of the period between
> v3 release and the day we make it a default version through consensus param.
> We ensure that we have proper testing.
> 
> Is your worry mostly about the uglyness of rolling back hidden service if that
> consensus param ever goes from v3 to v2? If so, I can understand how meh it
> is...

No, my main concern is that we can't validate the options on launch, if
we need a consensus to do it.

And what do we do if the options are valid for v2, but not v3, and the
HS downloads a consensus that says "use v3"?
Crash, probably, at least the way the code is written at the moment.

Here's a suggested strategy:
* at load time, validate the HS options as if v2 is the default, and
  validate them as if v3 is the default, and fail if either validation
  fails.
* then, act on the HS options as if v2 is the default, and also act as
  if v3 is the default, and fail if either action fails.
  (We need to do this because we don't discover some option issues
  until runtime, such as whether the directory can be created.)
* then, when each consensus is downloaded, publish whichever descriptor
  is the default in the consensus (if the HS config does not specify
  a specific version).

This means creating some redundant keys, but that's ok.

There is still a minor issue with the hostname file.
We could add two new files:
* hostname3: the hostname of the v3 hidden service
* hostname_published: the hostname of the published hidden service
                      (only set when the HS is actually published)

I think this is better for backwards-compatibility than changing the
content of the hostname file.

T

-- 
Tim Wilson-Brown (teor)

teor2345 at gmail dot com
PGP C855 6CED 5D90 A0C5 29F6 4D43 450C BA7F 968F 094B
ricochet:ekmygaiu4rzgsk6n
xmpp: teor at torproject dot org
------------------------------------------------------------------------