Proposal 158: Clients download consensus + microdescriptors

[Paul Syverson]

On Wed, Jan 21, 2009 at 12:46:04AM -0500, Nick Mathewson wrote:
> On Sun, Jan 18, 2009 at 02:22:52PM -0500, Roger Dingledine wrote:

> The reason for doing this particular design (call it "caches build
> microdescriptors") is mainly b above, so that if more info needs to
> get added to microdescriptors, the caches can just serve it, and they
> don't need to be upgraded.
> 
> The alternative would be to have the authorities generate and serve
> the microdescriptors themselves.  (Call this "authorities build
> microdescriptor".)  This would give us greater freedom in what we put
> in them and how we format them, but would require more stuff to get
> downloaded and cached from the authorities by the mirrors.
> 
> Even though I initially advocated it, I am not sure that the approach
> this proposal takes actually helps forward compatibility.  After all,
> the only reason to add a new field to microdescriptors is because
> clients are going to start using it.  So, clients are upgrading
> anyway.  Authorities would in both cases need to be reconfigured, at
> least, to vote for the new microdescriptor constructions.
> 
> So how are these approaches different in their outcomes?
> 
> Advantages for "Caches build microdescriptors"
> 
>    + Caches download less from the authorities. 
>    + It is trivial to determine that the authorities have computed the
>      microdescriptor for a descriptor correctly if you have both.
>      (With "Authorities build microdescriptors", you need to know the
>      algorithm that the authorities used, so it's easy, but not
>      quite so trivial.)
>    +??? We can change what goes into microdescriptors just by
>      upgrading the authority configuration.  This is not so great as
>      it might first seem.  Revisions to microdescriptor contents
>      shouldn't happen lightly, after all, since making them bigger
>      will defeat their purpose, and taking things out will make old
>      clients stop working.  Since changing their contents would
>      require a proposal and a client behavior shift anyway, is having
>      the authorities upgrade to a new consensus-version such a big deal?
> 
> Advantages for "Authorities build microdescriptors":
>    + We have more flexibility about what the microdescriptors can
>      contain.  For instance, they can't include the equivalent of the
>      "p" lines in the current consensus format, even though those need
>      to be calculated from exit policies, and are not simple copies.
>      This is especially important if our goal is to shift stable info
>      into the microdescriptors in order to keep consensuses small
>      while making clients download descriptors less.
> 
> That's 3 advantages for "Caches build", and only 1 for "Authorities
> build", but I think that the advantage of "authorities build" is much
> bigger.  It lets us consider things like the exit-ports line, binary
> packing of onion keys [not actually a win, but the next thing could
> be], and so on.  What do you think?
> 

Just a high-level support for Nick's point. The entire purpose here is
to cleverly eek out some much-needed/desired optimization of
distributing (micro)descriptor information, and as Roger (Needham)
always used to quote to us from Strachey, it's impossible to foresee
the consequences of being clever. In this case that means that until
this has had a chance to work through both in the spec and in
deployment experience for a while you probably want to be maximally
flexible so that an unforeseen change to microdescriptors that
ultimately emerges as necessary or at least clearly desirable is not
locked out or at least is not as painful or requiring of compatibility
tradeoff frustrations in decisions.

Another caveat about the pain of possibly not getting it perfectly
right the first time is potential attacks that somehow sneak in
between the job of the authorities and the job of the descriptors. No,
I don't have anything specific in mind, but its just an observation
that there is probably more protocol-level recovery flexibility (less
need to get it right the first time) in the authorities-build
approach. That shouldn't stop exploration, but the risk tradeoff
should be in mind at all times going forward.

aloha,
Paul