[tor-dev] [RFC] Directory structure of prop224 onion services

[s7r]

Hi,

David Goulet wrote:
> On 30 Jan (16:16:07), George Kadianakis wrote:
>> David Goulet <dgoulet at ev0ke.net> writes:
>>
>>> On 26 Jan (15:05:26), George Kadianakis wrote:
>>>> Hey list,
>>>
>>> Hi!
>>>
>>> First, big thanks for this write up!
>>>
>>>>
>>>> with service-side prop224 implementation moving forward, we need to pin down
>>>> the directory structure of prop224 onion services. This will be very similar to
>>>> the current directory structure, but with some mods to facilitate assymetric
>>>> client authorization keys and offline keys.
>>>>
>>>> As people have pointed out, the HS directory structure matters less after the
>>>> introduction of ephemeral ADD_ONION onion services, but still it's an important
>>>> part of onion service sysadmin UX.
>>>>
>>>> So the HiddenServiceDir directory will contain the following items:
>>>>
>>>> - "./hostname"    [FILE]
>>>>
>>>>    This is a file containing the onion address of the onion service.
>>>>
>>>>    As you can see it's the same filename as in v2. Should we suffix it with v3
>>>>    to make it clear that it's v3 onion? Would we ever have v2 and v3 onions
>>>>    living in the same directory?
>>>
>>> I don't believe we should suffix here because for almost 10 years, users/apps
>>> have been exposed to "hostname" and it does make sense that it's the goto file
>>> for that.
>>>
>>> Current implementation doesn't allow two services in the same HiddenServiceDir
>>> and for prop224, the ongoing implementation doesn't allow it either. Sharing a
>>> directory brings all sorts of uneeded complexity. So if the directory is v3,
>>> everything in it will be v3.
>>>
>>>>
>>>> - "./private_key_ed25519"  [FILE]
>>>>
>>>>    This is the file containing the private master ed25519 key of the onion service.
>>>>
>>>>    If offline keys are _enabled_, then this file doesn't exist and instead a
>>>>    directory is made containing blinded keys for every day [TODO: The directory
>>>>    format here will be specified in the future].
>>>
>>> If that file doesn't exists, the public key is needed else the service can't
>>> derive the .onion and create the hostname file. The offline case is an extra
>>> use case but I suspect we would use "public_key_ed25519" along with the
>>> blinded keys specific file name. (Unless we make our "tor-genkey" tool
>>> generate the hostname file as well. #bikesheding)
>>>
>>>>
>>>> - "./client_authorized_pubkeys"   [FILE]
>>>>
>>>>   If client authorization is _enabled_, this is a newline-separated file of
>>>>   "<client name> <pubkey>" entries for authorized clients. You can think of it
>>>>   as the ~/.ssh/authorized_keys of onion services.
>>>>
>>>> - "./client_authorized_privkeys/"          [DIRECTORY]
>>>>   "./client_authorized_privkeys/alice"     [FILE] 
>>>>   "./client_authorized_privkeys/bob"       [FILE]
>>>>   "./client_authorized_privkeys/charlie"   [FILE]
>>>
>>> Small clarification. The "<client name>" field in the the pubkey file is the
>>> same for the privkey file name. So if "alice" is in the pubkey file, it will
>>> be "alice" in this privkey directory.
>>>
>>>>   
>>>>   If client authorization is _enabled_ _AND_ if the hidden service is
>>>>   responsible for generating and distributing private keys for its clients,
>>>>   then this directory contains files with client's private keys. The idea is
>>>>   that these files can be shredded and deleted after the private key has been
>>>>   passed to the client. For more context here, please read the client
>>>>   authorization thread in [tor-dev] and see 'Appendix F' of prop224 for more
>>>>   details on how this works.
>>>
>>> Also, expected behavior that we should go for when implementing this within
>>> the "tor" code base. We could think of many ways to make this more complex
>>> that it could be but going *simple* is what I'm aiming for:
>>>
>>> - The torrc option HiddenServiceAuthorizeClient as to match the list of client
>>>   in the pubkey file in so if the pubkey file has extra entries, we error at
>>>   startup. With this in mind, here are the behaviors:
>>>
>>> i) if a privkey file exists but no entry in the pubkey file, add the entry to
>>>    pubkey file as long as the client name is found in
>>>    HiddenServiceAuthorizeClient.
>>>
>>> ii) a pubkey entries does NOT need a corresponding privkey to be used. As long
>>>     as the client name is found in HiddenServiceAuthorizeClient.
>>>
>>> iii) if a client name is specified in the HiddenServiceAuthorizeClient option but
>>>      NOT in the pubkey file, generate pubkey/privkey unless the privkey file
>>>      exists which is (i) behavior where pubkey is derived from privkey.
>>>
>>> So the great thing about this is that you can create a keypair on a different
>>> machine (or client side), put the privkey file in the
>>> client_authorized_privkeys directory and add the client name to torrc, HUP tor
>>> and done. We could see ultimately an auto update of the service configuration
>>> with the client name but I'm not a big fan of changing the torrc file
>>> automagically...
>>>
>>
>> Hey David,
>>
>> thanks for the useful comments.
>>
>> Please check my torspec branch `prop224-directory-format`.
>>
>> FWIW, I agree with all the expected behavior details you noted at the
>> end of your email. I encoded some of those behaviors in the spec, but I
>> didn't provide a complete formal algorithm of how the whole process
>> works because I don't think it's spec material and also because I feel
>> that during implementation we will get new insights on how this should
>> work.
>>
>> Let me know how you feel about the spec patch :)
> 
> Good stuff! And yes, I don't think it's spec material at all but good to have
> in an Appendfix for reference. Once this file structure will be released in a
> tor version, we *must* update the man page FILES section.
> 

Directory structure looks good to me.

I agree the file containing the address should just be called 'hostname'
with no version suffix or anything, it will be redundant and create
confusions as soon as v2 is entirely deprecated. Also, 'hostname' file
is only responsible to provide the right address that can be used to
connect to the onion service, it should not care about the version or
include this information it the containing file name.

What we will do here to avoid accidents is that we will use the
public_key_ed25519 file as the starting point. If this file is present,
we will assume the onion service is generated and existent. If not,
compute it and save to disk either from private_key_ed25519 (if
available) or the blinded keys (if available).

I like the protocol suggested by David very much for managing auth keys
for clients. Nice, logic and should leave no room for accidents.

Here's another protocol for managing the onion service identity (keys).
This is not urgent but I am writing it now so I won't forget something
and we will have it here when needed:

1. Check if private_key_ed25519 exists. If it doesn't exist, move to
step 2.
1.1 If it exists, compute public_key_ed25519 and hostname, or validate
the latter two if they exist. If there is a mismatch, exit with error,
user accidentally mixed something.

2. Check if public_key_ed25519 exists. If it doesn't exist, move to step 3.
2.1 If it exists, compute hostname or validate the latter one if it
exists, If there is a mismatch, exit with error, used accidentally mixed
something.

3. Check if a blinded key exists.
3.1 If it doesn't exist, and private_key_ed25519 is available, compute
it and go to step 3.2, otherwise go to step 4.
3.2 If it exists and is valid, compute public_key_ed25519 and hostname,
or validate the latter two if they exist. If there is a mismatch, exit
with error, user accidentally mixed something.
3.3 If it is invalid and private_key_ed25519 is available, compute a new
one and go back to step 3.2, otherwise exit with error asking for manual
further action with genkey tool.

4. If offline onion service keys is not set, create a new
private_key_ed25519 and go to step 1 again.
4.1 If offline onion service keys is set, exit with error asking for
manual further action with genkey tool.


As always, epic work. Thanks !

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20170131/a7d0c86b/attachment.sig>