[tor-onions] Garbage collection of abandoned onion servers

[p13dz p13dz]

On Thu, Feb 9, 2023 at 11:12 PM rhatto <rhatto at torproject.org> wrote:

> On Sun, Dec 25, 2022 at 08:14:30PM +0100, p13dz p13dz via tor-onions wrote:
> >    Hello,
> >    We've built a solution to scale our onion server horizontally using
> >    Kubernetes horizontal autoscaler. It's basically an onionbalance &
> >    a set of wrappers to regenerate its config file depending on the set
> of
> >    running TorHiddenService instances.
>
> Very interesting!
>
> I wonder if you plan to release this solution as a free/open source
> software at some point.
>

That's definitely possible. We'd need to separate it from the rest of our
product, clean everything up, write docs, etc.

As a side note, after running our solution in production under a few
(D?)DoS attacks, we found some downsides to it (and to the Onion system
limits in general):

   - Very low limit on the descriptor size in the Onion protocol. Every
   server instance needs to have its own entry in the descriptor, and at 7-10
   instances the descriptor gets saturated, which makes further scaling up
   impossible.
   - The current implementation of onionbalance doesn't support automatic
   adjustment of the number of intro points per server instance. As an
   example, when I have 1 server instance running, I might want to have 6
   intro points to make sure they don't become an easy target for DoS. When we
   scale up the service, say 3x, 3*6=18 intro points no longer fit in the
   descriptor, and onionbalance breaks down and stops publishing updates at
   all. It could be smart enough to automatically lower the "replication
   factor" until the message fits in the descriptor. If the number of
   instances is too high, it should publish at least some of them instead of
   nothing.

At the level of the Onion protocol, it would be great to increase the
number of intro points per descriptor to a higher number (say 10x from what
we have today).
If the Tor team could assist with that, we'd publish our autoscaler and
together make Tor a cool platform for building services resilient to
serious attacks.
Let me know what you think.


>
> >    Each time we scale the set of servers up, we start new Tor onion
> >    servers on random addresses (that's what onionbalance expects), and
> >    their descriptors are then merged into the single one by onionbalance.
> >    When we delete old instances, their onion addresses are abandoned.
> >    We want to be good citizens and not to clutter Tor's DHT with
> abandoned
> >    records. My question basically is whether the Tor ecosystem
> >    automatically deletes abandoned descriptors or not. If yes, we can
> keep
> >    our system simpler. If not, we'll have to make the system stateful and
> >    try to reuse our existing onions as much as possible.
>
> TL;DR I think you can keep your system simple :)
>
> Please others correct if I'm wrong, but as far as I understand you don't
> have to worry about having disposable Onion Service addresses.
>
> This does not overload Tor's "DHT-like" HSDir structure (which in fact is
> a hash ring in constant renewal).
>
> From the [rend-spec-v3]:
>
>     2.2.2. When to publish a hidden service descriptor [WHEN-HSDESC]
>
>        Hidden services periodically publish their descriptor to the
> responsible
>        HSDirs. The set of responsible HSDirs is determined as specified in
>        [WHERE-HSDESC].
>
>        Specifically, every time a hidden service publishes its descriptor,
> it also
>        sets up a timer for a random time between 60 minutes and 120
> minutes in the
>        future. When the timer triggers, the hidden service needs to
> publish its
>        descriptor again to the responsible HSDirs for that time period.
>
>     [...]
>
>     2.2.5. Expiring hidden service descriptors [EXPIRE-DESC]
>
>        Hidden services set their descriptor's "descriptor-lifetime" field
> to 180
>        minutes (3 hours). Hidden services ensure that their descriptor
> will remain
>        valid in the HSDir caches, by republishing their descriptors
> periodically as
>        specified in [WHEN-HSDESC].
>
> So a descriptor vanishes from the HSDirs after a while if it's not
> regularly published. (See also section [HASHRING] and other sections in
> that document for more info).
>
> [rend-spec-v3]:
> https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/rend-spec-v3.txt


That's perfect, thanks!


>
>
> --
> Silvio Rhatto
> pronouns he/him
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-onions/attachments/20230211/496105ec/attachment.htm>