[tor-dev] [Draft Proposal] Scalable Hidden Services

[Matthew Finkel]

On Mon, Oct 28, 2013 at 12:53:35PM -0400, Nick Mathewson wrote:
> On Mon, Oct 28, 2013 at 9:19 AM, Matthew Finkel
> <matthew.finkel at gmail.com> wrote:
> > Hi everyone,
> >
> > This is a proposal I wrote to implement scalable hidden services. It's
> > by no means finished (there are some slight inconsistencies which I will
> > be correcting later today or tomorrow) but I want to make it public in
> > the meantime. I'm also working on some additional security measures that
> > can be used, but those haven't been written yet.
> >

I forgot to note that it is in the hs_scaling branch.

> > Many thanks to George for his initial feedback. I pushed this version to
> > my public repo, and will continue to push updates there.
> >
> > In reality this is really 3.2 proposals, so the end result, if accepted,
> > will look significantly different, but I'm indecisive at this point
> > about which of the designs is least dangerous.
> 
> Looks cool!
> 

I'm glad you think so!

> Generally, I think that requiring a single node to be master is fine
> in initial designs only: if we consider such a design, it's important
> to have a migration path to a design where the whole service doesn't
> fall over when a single leader goes down.  It's less important to me
> that there be no master key, especially if that master key can be kept
> offline.
> 

I agree. To be honest, the master-slave design is included more for
completeness and comparison than for implementation. I think it is a
reasonable design, but I favor much more the peer-nodes design, in
general.

I also decided to propose a simpler method of selecting a captain. The
original process resulted in a random node being selected, but I
decided it was unnecessarily complex, at this point.

Allowing offline key storage would be ideal, and with inter-node
communication a key can easily be distributed to the other nodes, as
needed. But, depending on the design, it really only buys a hidden
services operator the ability to upload a key on one node and have it
automagically distributed to the others, whereas the operator would need
to do this manually otherwise.

> I also like designs where it's not immediately obvious how many hosts
> a hidden service has just from looking at its descriptor.
> 

This would also be ideal, however after spending some time working on
this problem I don't like the currently available solutions for solving
this. At its core, we are trying to take a one-to-one mapping and turn
it into a one-to-many. The proposed solutions are more like hacks, if
this is to be done correctly I think we should try to redesign the
components that are causing problems instead of trying to mangle the
current design in a (hopefully) secure way.

> 
> FWIW, I'm working on a complete revision to rend-spec.txt, since I
> think the scope of hidden service changes under consideration is broad
> enough that it makes sense to reconsider the entire system as a whole
> rather than trying to do it one piece at a time.  I haven't even
> gotten to the spec parts yet -- just the preliminaries -- but you can
> see the draft in progress as it was on my desktop this morning in
> branch "rendspec-ng" in my torspec repo.  I'll be putting more updates
> there as I go.

Great, I'll take a look. I think this will result in a better (more
secure, usable, cohesive) product.

> 
> I mention that here because some of the design work I'm recording
> there should mitigate the dangers of distributing keys to different
> hosts.
> 

Oh, this reminds me, I forgot to thank George for his suggestion to use
the Key Transport protocol (thanks again George!).

Mitigating the danger will be nice. I'm also not sure how severe the
danger is or if a secret sharing scheme is really necessary for this,
but I added it in any case for the defense in-depth approach.

Thanks for your thoughts/comments!

- Matt