Proposal 158 revised: Microdescriptors again

[Nick Mathewson]

[I've snipped the points where I agree with you and am just merging
your changes into the proposal.]

On Sat, Jun 13, 2009 at 03:23:41AM -0400, Roger Dingledine wrote:
> On Sat, May 16, 2009 at 12:46:40AM -0400, Nick Mathewson wrote:
 [...]
> > 3.1.2. Computing consensus for microdescriptor-elements and "m" lines
> > 
> >   When we generating a consensus, we use whichever m line
> >   unambiguously corresponds to the descriptor digest that will be
> >   included in the consensus.  (If there are multiple m lines for that
> >   descriptor digest, we use whichever is most common.  If they are
> >   equally common, we break ties in the favor of the lexically
> >   earliest.  Either way, we should log a warning: That's likely a
> >   bug.)
> 
> I don't understand the above. The microdescriptor is a straight
> function of a) the consensus method, and b) whichever relay descriptor
> is a winner for this vote. So that means the "m" line we pick for the
> consensus has to be the microdescriptor digest from whichever votes a)
> voted for the winner, and b) offered the consensus method that we chose
> to use for constructing this consensus. If any such votes differ, we'll
> have other problems, like disagreeing authorities serving disagreeing
> microdescriptors. I guess that's what you're getting at above?

Right.  If two authorities cannot agree on which microdescriptor is
produces by transforming a descriptor with digest X with consensus
method Y, then (at least) one of them is broken.  But we need to keep
going in this case, since we don't want a single broken authority to
be able to prevent us from forming a consensus.  So we specify that
ties are broken by going with the most common digest, followed by the
lexically earliest.

 [...]
> >   All the microdescriptors from the current consensus should also be
> >   available at:
> >     http://<hostname>/tor/micro/all.z
> 
> How will these two URLs interact with future flavors? If a later flavor
> uses a different hash function, do we still offer everything under
> /tor/micro/d/<D>, even though different clients are verifying results
> with different hash functions?

According to proposal 162, flavors are supposed to apply to
consensuses.  There is no current design for multiple parallel flavors
of microdescriptors.  So I guess we need to figure this out.

Thinking aloud.... Let's consider two cases:

    A) A new flavor supports a new hash algorithm, and refers to
       microdescriptors by this hash.  Clients want to download
       microdescriptors by this hash, but they are the same as the
       other microdescriptors.

    B) A new flavor has its own notion of microdescriptors, and these
       microdescriptors are not the same as those generated with some
       other flavor of consensus.

I think that A and B want different solutions.  For A, caches should
download microdescriptors by whichever hash algorithm they like best,
and serve them by all the hash algorithms they know.  Because caches
can only use flavors they understand, I think that we can safely defer
specifying how they handle this case until there _are_ two such
flavors out there.

Right now, we have no notion or design for B at all.  My intuition
says that we should try to avoid B, and if it happens, we have created
something that isn't a microdescriptor.

Or we could try to do a bit like you suggest, so that there is
     /tor/micro/MF/d-sha256/<D1>
where MF is a "microdescriptor flavor" such that multiple flavors of
consensus can share a single microdescriptor flavor.

 [...]
> If we do change our hash function, caches that don't recognize the flavor
> won't be able to verify that the microdescriptor hashes are correct. And
> since they won't know whether we've changed the hash function in a
> flavor they don't recognize, does that mean that caches should never
> check hashes on flavors they don't recognize?

IMO if you have no way to check a hash, you should not be serving
documents indexed only by that hash.  So if a cache doesn't support
AHS, it shouldn't ever serve /tor/micro/MF/d-ahs/<D1> .  Anything else
seems crazy to me.

Another issue with microdescriptors: unlike router descriptors, there
isn't a trivial way to determine microdescriptor boundaries when you
get a lot of them concatenated.  I suppose we could introduce an
element that always comes first, or say that they're separated by
blank lines, or something like that.

yrs,
-- 
Nick