[tor-dev] [DRAFT] Proposal: All Routers are Directory Servers

[Matthew Finkel]

Hi All,

Below is a draft proposal for making all relays also be directory
servers (by default). It's almost ready for a number, but it can use
some feedback beforehand (give or take a few days).

Tonight I also found that Nick actually created a similar proposal
a few moons ago (Proposal 185: Directory caches without DirPort). This
new proposal may benefit from superseding that one, but I recommend they
remain distinct proposals, for now.

In general I'm not tied to any of the proposed names, so suggestions
are welcome.

Thanks!


=======================================================================


Filename: directory-servers-for-all.txt
Title: All routers are directory servers
Author: Matthew Finkel
Created: 29-Jul-2014
Status: Draft
Target: 0.2.6.x

Overview:

    (In practice we refer to the servers that mirror directory documents
as directory servers, directory mirrors, and directory caches. Sometimes
we also shorten directory to dir. This proposal attempts to consistently
use directory server to refer to this functionality. We also typically
use the term router and relay interchangably. This proposal uses (onion)
router.)

    This proposal aims at removing part of the distinction between the
router and the directory server. Currently operators have the options
of being one of: an onion router, a directory server, or both.  With the
acceptance of this proposal the options will be simplified to being
either only a directory server or a combined router and directory
server. All routers will serve directory requests.

Motivation:

    Fetching directory documents and descriptors is sometimes a
non-trivial operation for clients. If they do not have a consensus then
they must contact a directory authority (until directory sources are
added or clients are able to use a fallback consensus). If they have a
consensus and have at least one entry guard then the client can query
that guard for documents. If the document isn't available then after a
period of time the client will attempt to retry downloading it. If the
entry guard isn't a directory server, as well, a directory server and/or
directory guard must be chosen (based on the server having an open
DirPort) and queried for the document. At a minimum, this has a
potential performance impact, at worst it's an attack vector that
allows for profiling clients and partitioning users. With the
orthogonally proposed move to clients using a single guard, the
potential performance bottleneck and ability to profile users could be
increased.

    If the client selects an entry guard and it is not a
directory server then the client may select a distinct directory guard
which will leak client behavior to a second node. In the case where the
client does not use guards, it is important to have the largest possible
amount of diversity in the set of directory servers. In a network where
every router is a directory server, the profiling and partitioning
attack vector is reduced to the guard (for clients who use them), which
is already in a privileged position for this. In addition, with the
increased set size, relay descriptors and documents are more readily
available and it diversifies the providers.


Design:
    
    The changes needed to achieve this should be simple. Currently all
routers download and cache the majority of router documents in any case,
so the slight increased memory usage from downloading all of them should
have minimal consequences. There will be necessary logical changes in
the client, router, and directory code.
    
    Currently directory servers are defined as such if they advertise
having an open directory port. We can no longer assume this is true. To
this end, we will introduce a new server descriptor line.

	"tunnelled-dir-server"

    The presence of this line indicates that the router accepts
tunnelled directory requests. For a router that implements this
proposal, this line MUST be added to its descriptor if it does not
advertise a directory port, and MAY be added if it also advertises an
open directory port. In addition to this, routers will now download and
cache all descriptors and documents listed in the consensus, regardless
of whether they are deemed useful or usable, exactly like the current
directory servers. All routers will also accept directory requests when
they are tunnelled over a connection established with a BEGIN_DIR cell,
the same way these connections are already accepted by bridges and
directory servers with an open DirPort.

    Directory Authorities will now assign the V2Dir flag to a server if
it supports a version of the directory protocol which is useful to
clients and it has at least an open directory port or it has an open OR
port and advertises "tunnelled-dir-server" in its server descriptor.

    Clients choose a directory by using the current criteria with the
additional qualification that a server only needs the V2Dir status flag
instead of requiring an open DirPort. When the client chooses which
directory server it will query, it checks if the server has an open
directory port and uses begindir if it does not. This will minimize the
increased number of clients that prefer begindir over direct connections
and thus also minimizes the additional overhead on the network from
clients establishing OR connections for directory requests.

Security Considerations and Implications:

    Currently all directory servers are explicitly configured. This is
necessary because they must have a configured and reachable external
port.  However, this is a restriction and results in a reduced number of
directory servers on the network. As a result, this could allow an
adversary to control a significant fraction of the servers. By
increasing the number of directory servers on the network the likelihood
of selecting one that is malicious is reduced. Also, with this proposal,
it will be more likely that a client's entry guard is also a directory
server (as alluded to in Proposal 207). However, the reduced anonymity
set caused when the guard does not have, or is unwilling to distribute,
a specific document still exists. With the increased diversity in the
available servers, the impact of this should be reduced.

    Another question that may need further consideration is whether we
trust bad directories to be good guards and exits.

Specification:

    The version 3 directory protocol specification does not
currently document the use of directory guards. This spec should be
updated to mention the preferred use of directory guards during
directory requests. In addition, the new criteria for assigning the
V2Dir flag should be documented.

Considerations for Resource Constrained Hardware:

    If all routers become directory servers, they will choose to
download all documents, regardless of whether they are useful, in case
another client does want them. This will have very little impact on the
"typical" router, however on memory constrained routers (Raspberry Pi
and similar), every kilobyte allocated to directory documents is not
available for new circuits. Should we add a config option that allows
operators to disable being a directory server?  Is it more worthwhile
for them to serve these documents or to relay cells?

Future Considerations:

    Should the DirPort be deprecated at some point in the future?