[tor-dev] Tor's default behavior for ed25519 identities

[s7r]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 8/4/2015 5:42 PM, Nick Mathewson wrote:
> Hi, s7r!
> 
> This is an impressive writeup; thanks!
> 
> One thing that makes it hard for me to follow this document is
> that I'm not sure which parts are describing how things work _now_,
> and which parts describe how things _should_ work.  Would it be
> possible to split up descriptions of current/revised behavior, and
> mark each?
> 
> many thanks,
> 

Hi Nick,

My pleasure. Done and written. It looks long but I hope it reads fast
and makes sense. I know it's a PITA to fix things by reading about
them on email, so do let me know what are the priorities and how
should I open tickets on trac to be cleaner and easier for you (a
single big ticket? more tickets for each case to the master ticket
where we discuss the Ed25519 FAQ?).

I have assigned status flags to cases as OK (already doing what it
should), NOT OK (mostly doing what it should, needs just little
adjustments), DEFECT (really broken behavior).

One thing which I've noticed is that Tor thinks very highly about
ed25519_master_id_public_key, it treats it like the identity key
itself (and I don't know why, since it can compute this from
ed25519_signing_cert?) - Tor will go as far as generating a new
identity if we take ed25519_master_id_public_key away. See each case
documented below.

Note: I was only able to use restart (not reload signal [HUP]) because
it doesn't work for me after playing with ed25519 key files ( reported
in https://trac.torproject.org/projects/tor/ticket/16703 )


1. No ed25519* files at all in $datadirectory/keys:

What Tor should do:
- - We assume it's a new relay or a relay which just upgraded to a Tor
that supports Ed25519 router identities. Generate a new Ed25519
identity unencrypted (ed25519_master_id_secret_key,
ed25519_master_id_public_key, ed25519_signing_secret_key and
ed25519_signing_cert with a validity for 30 days unless specified
otherwise in torrc) and publish a descriptor using that, proceed as
normal after.

What Tor currently does:
- - Generates a new ed25519 identity unencrypted
(ed25519_master_id_secret_key, ed25519_master_id_public_key,
ed25519_signing_secret_key and ed25519_signing_cert with a validity of
30 days, unless specified otherwise in torrc).

Status: OK
- --------------------------------------------------------------------

2. If ed25519_master_id_secret_key is missing or encrypted and
ed25519_master_id_public_key exists in $datadirectory/keys (without
ed25519_signing_secret_key and ed25519_signing_cert):

What Tor should do:
- - Don't start at all, something is wrong. Print warnings that user
action is required (either generate valid ed25519_signing_secret_key
and ed25519_signing_cert either move the ed25519_master_id_secret_key
unencrypted in $datadirectory/keys so Tor can do it by itself).

What Tor currently does:
Doesn't start:
Aug 04 10:53:36.000 [warn] Missing identity key
Aug 04 10:53:36.000 [err] do_main_loop(): Bug: Error initializing
keys; exiting (on Tor 0.2.7.2-alpha-dev )

Status: NOT OK. Behavior is correct. We should just change the log
message with a better, more explicit one and remove 'Bug' (it isn't a
bug).
- --------------------------------------------------------------------

3. If ed25519_master_id_secret_key unencrypted and
ed25519_master_id_public_key both exist in $datadirectory/keys and
ed25519_signing_secret_key, ed25519_signing_cert are missing:

What Tor should do:
- - Assume it's a reinstalled relay with backups of the identity.
Generate valid ed25519_signing_secret_key and ed25519_signing_cert
(with a validity period of 30 days unless specified otherwise in
torrc) and start, proceed as normal after.

What Tor currently does:
- - Generates valid ed25519_signing_secret_key and ed25519_signing_cert
(with a validity period of 30 days unless specified otherwise in
torrc), starts and proceeds as normal.

Status: OK
- --------------------------------------------------------------------

4. If only ed25519_master_id_secret_key unencrypted exists in
$datadirectory/keys (ed25519_master_id_public_key,
ed25519_signing_secret_key and ed25519_signing_cert are missing):

What Tor should do:
- - We assume the relay has just been restored and the Ed25519 identity
was backed up. Automatically generate ed25519_master_id_public_key,
ed25519_signing_secret_key and ed25519_signing_cert with a validity
period of 30 days (unless otherwise specified in torrc).

What Tor currently does:
- - It starts and generates valid ed25519_signing_secret_key and
ed25519_signing_cert first time but DOES NOT generate
ed25519_master_id_public_key. Starts fine first time.

After a service tor restart, it dies:
Aug 04 11:45:46.000 [warn] The signing cert we have was not signed
with the master key we loaded!
Aug 04 11:45:46.000 [err] do_main_loop(): Bug: Error initializing
keys; exiting (on Tor 0.2.7.2-alpha-dev )

Here is what Tor is actually doing:
- - It sees the valid unencrypted ed25519_master_id_secret_key file and
generates ed25519_signing_secret_key and ed25519_signing_cert because
these files are missing. OK. but it doesn't check or care about
ed25519_master_id_public_key which is wrong. Starts.
- - After first restart, because it sees the ed25519_signing_secret_key
and ed25519_signing_cert proceeds further to validation and notices
that ed25519_master_id_public_key is missing. Instead of computing it
from the ed25519_signing_cert, it generates an entire new identity
(ed25519_master_id_secret_key file [yes, it will overwrite the old
existent file and change the identity forever] and
ed25519_master_id_public_key file).
- - Now it insanely checks the old ed25519_signing_secret_key and
ed25519_signing_cert with the new generated
ed25519_master_id_secret_key and ed25519_master_id_public_key in an
endless loop - obviously they cannot validate, so it dies like this:
[warn] The signing cert we have was not signed with the master key we
loaded!

No matter how many restarts after this, it will still say:
[warn] The signing cert we have was not signed with the master key we
loaded!

Only way to start Tor now is to delete manually
ed25519_signing_secret_key and ed25519_signing_cert from
$datadirectory/keys (Tor will generate valid new ones for the correct
new unrequested identity) or delete the new unrequested
ed25519_master_id_secret_key and ed25519_master_id_public_key and
bring back the old ed25519_master_id_public_key which validates with
the ed25519_signing_secret_key and ed25519_signing_cert.

Status: DEFECT. Order of operations is not logic. Tor should check for
existence of files, generate what is missing and can be generated,
validate, die or proceed.
- --------------------------------------------------------------------

5. If only ed25519_master_id_secret_key_encrypted exists in
$datadirectory/keys (ed25519_master_id_public_key,
ed25519_signing_secret_key and ed25519_signing_cert are missing):

What Tor should do:
- - We assume the relay identity existed previously and
ed25519_master_id_secret_key_encrypted was restored form a backup. Do
not start Tor at all and print explicit warnings that action is
required - Tor cannot decrypt the
ed25519_master_id_secret_key_encrypted file. Generate
ed25519_master_id_public_key, ed25519_signing_secret_key and
ed25519_signing_cert or decrypt the
ed25519_master_id_secret_key_encrypted in $datadirectory/keys so Tor
can do it by itself.

What Tor currently does:
- - Ignores the ed25519_master_id_secret_key_encrypted and generates
entire new identity + signing key and signing cert and starts as normal.

Status: DEFECT. If there is an ed25519_master_id_secret_key_encrypted
in $datadirectory/keys we never generate automatically another new
identity, instead we push the user to decrypt it or manually generate
valid ed25519_signing_secret_key and ed25519_signing_cert.
- --------------------------------------------------------------------

6. If ed25519_master_id_secret_key_encrypted and
ed25519_master_id_public_key both exist in $datadirectory/keys
(ed25519_signing_secret_key and ed25519_signing_cert are missing):

What Tor should do:
- - We assume the relay identity existed previously and
ed25519_master_id_secret_key_encrypted was restored form a backup. Do
not start Tor at all and print explicit warnings that action is
required - Tor cannot decrypt the
ed25519_master_id_secret_key_encrypted file. Generate
ed25519_master_id_public_key, ed25519_signing_secret_key and
ed25519_signing_cert or decrypt the
ed25519_master_id_secret_key_encrypted in $datadirectory/keys so Tor
can do it by itself.

What Tor currently does:
- - It does not start at all because it doesn't have a valid
ed25519_signing_secret_key and ed25519_signing_cert:
Aug 04 11:55:20.000 [warn] Missing identity key
Aug 04 11:55:20.000 [err] do_main_loop(): Bug: Error initializing
keys; exiting (on Tor 0.2.7.2-alpha-dev )

Status: NOT OK. Behavior is correct. We should just change the log
message with a better, more explicit one and remove 'Bug' (it isn't a
bug).
- --------------------------------------------------------------------

7. If ed25519_master_id_secret_key unencrypted and valid
ed25519_signing_secret_key and ed25519_signing_cert exist in
$datadirecotry/keys (ed25519_master_id_public_key is missing):

What Tor should do:
- - Compute ed25519_master_id_public_key from ed25519_signing_cert (in
order to keep a logic), validate and start. Proceed as normal after
(regenerate automatically after expiration if master key unencrypted
still available).

What Tor currently does:
- - Generates a new identity (ed25519_master_id_secret_key [yes, it
overwrites the old identity key file] and
ed25519_master_id_public_key) and tries to validate the existent
ed25519_signing_secret_key and ed25519_signing_cert with the new
unrequested identity - obviously it fails and dies (see Case 4 above):

Aug 04 10:57:43.000 [warn] The signing cert we have was not signed
with the master key we loaded!
Aug 04 10:57:43.000 [err] do_main_loop(): Bug: Error initializing
keys; exiting (on Tor 0.2.7.2-alpha-dev )

It becomes an endless loop, and won't ever start, always tries to
validate the old cert and key with the new identity. Only way to start
is to delete signing key and cert or delete the new unrequested
identity and bring back the old ed25519_master_id_public_key.

Status: DEFECT. Order of operations is not logic. Tor should check for
existence of files, generate what is missing and can be generated,
validate, die or proceed.
- --------------------------------------------------------------------

8. If only ed25519_master_id_public_key and valid
ed25519_signing_secret_key and valid ed25519_signing_cert exist in
$datadirectory/keys:

What Tor should do:
- - Start normally, all good.

What Tor currently does:
- - Starts normally, all good.

Status: OK
- --------------------------------------------------------------------

9. If only valid ed25519_signing_secret_key and valid
ed25519_signing_cert exist in $datadirectory/keys
(ed25519_master_id_public_key missing):

What Tor should do:
- - Compute ed25519_master_id_public_key from ed25519_signing_cert (in
order to keep a logic), validate and start. Proceed as normal after
(regenerate automatically after expiration if master key unencrypted
is still available).

What Tor currently does:
- - Generates a new identity (ed25519_master_id_secret_key [yes, it
overwrites the old identity key file] and
ed25519_master_id_public_key) and tries to validate the existent
ed25519_signing_secret_key and ed25519_signing_cert with the new
unrequested identity - obviously it fails and dies (see Cases 4 and 7
from above).

Status: DEFECT. Order of operations is not logic. Tor should check for
existence of files, generate what is missing and can be generated,
validate, die or proceed.
- --------------------------------------------------------------------

10. If we have in $datadirectory/keys an ed25519_master_id_secret_key
and ed25519_master_id_public_key which doesn't belong to the secret key:

What Tor should do:
- - Don't start at all - obviously something is wrong. Print messages in
log file that user action is required, we need either the old identity
ed25519_master_id_secret_key unencrypted if it was backed up, either a
whole new identity (so delete all edd25519* files from
$datadirectory/keys and start fresh) either a valid
ed25519_signing_secret_key and valid ed25519_signing_cert).

What Tor currently does:
- - Doesn't start at all:
Aug 04 12:10:25.000 [warn]
/var/lib/tor/keys/ed25519_master_id_public_key does not match
/var/lib/tor/keys/ed25519_master_id_secret_key!
Aug 04 12:10:25.000 [warn] Missing identity key
Aug 04 12:10:25.000 [err] do_main_loop(): Bug: Error initializing
keys; exiting (on Tor 0.2.7.2-alpha-dev )

And it also doesn't generate a new unrequested identity.

Status: NOT OK. Behavior is correct, we just need to adjust the log
messages. Not a Bug.
- --------------------------------------------------------------------

11. Probably not a bug: key files have to be owned by the same user
running Tor, or they cannot be read (chmod is 0600 for all of them):

Aug 04 11:54:09.000 [warn] Could not open
"/var/lib/tor/keys/ed25519_signing_secret_key": Permission denied
Aug 04 11:54:09.000 [warn] Could not open
"/var/lib/tor/keys/ed25519_master_id_public_key": Permission denied
Aug 04 11:54:09.000 [warn] Unable to read
/var/lib/tor/keys/ed25519_master_id_public_key: Permission denied

####### THE END #######
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (MingW32)

iQEcBAEBCAAGBQJVwVehAAoJEIN/pSyBJlsR3PQH/iZx1tDQn36lYQ/+phZvITJE
wgntsxF2g/ET7BYEU4A8sdWwQN0BQIYLrrWcez6M7mwhV0MD9aW71QALFkTiBPa1
hXaX09UZwnCybl0GD/luM7NMrFcxb46Fk1ZWWJkr46ZVuetLGwDvP1XvfGnjaIfG
xTEZz79A8NXQw2WGjNPH0LaOKlRFjrTGsi/pajVOYYkhwTXA4zk6cgwn4vY6ODVl
ngnrlPJzGobFRvfXKYoiw7eCSiZdIW2Ind7i7iZr4nmoe3O+v66Kp16xeq9qTA4W
4+V5cWj5yhgTGcv9V7k6wfY4naI+iG0B4hf5fRPddKkW5NO3opDssnG0BHeKY4k=
=TcbH
-----END PGP SIGNATURE-----