[tor-talk] Massive Bandwidth Onion Services

[George Kadianakis]

Alec Muffett <alec.muffett at gmail.com> writes:

> I would post this to the tor-onions list, but it might be more generally
> interesting to folk, so I'm posting here and will shift it if it gets too
> technical.
>
>
> I'm working on load-balanced, high-availability Tor deployment
> architectures, and on that basis I am running 72 tor daemons on a cluster
> of 6 quad-core Debian boxes.
>
> I am then - using Donncha's "OnionBalance" to:
>
> * scrape the descriptors of those 72 daemons
>
> * selects random(ish) 60 of the introduction points from those daemons, and
>
> * rebundle those 60 introduction points into 6 distinct descriptors of 10
> introduction points apiece, then
>
> * signing those distinct descriptors with a "service" onion address and
> emplacing them onto the HSDir ring.
>
>
> This means that, at any one time, the daemon will be able to have 60x the
> CPU and network-bus bandwidth, above/beyond what is available to a typical
> single-daemon instance.
>
> Why "72"? Because it's a number >60 and I'm seeking to stress-test things a
> little.
>
> Specifically: one eventual goal of this project is to adjust the timings a
> little, so that the HSDir cache acts a little like "Round-Robin DNS Load
> Balancing" - people accessing the "service" onion address at lunchtime will
> receive/cache different descriptors from those who access it some hours
> later, and the descriptors persist, so thereby the whole 72 daemons get
> used/averaged-out over an entire day.
>
> In my attempts to go fast-and-wide, however, I appear to have run into a
> hardcoded limit:
>
>     Dec 19 09:24:43.365 [warn] HiddenServiceNumIntroductionPoints should be
> between 3 and 10, not 1
>
> There's little point in publishing >2, and perhaps* not >1 introduction
> point for each of the 72 daemons; also it makes scraping and reassembly
> slower/more expensive.
>
> So I am writing to ask whether it is possible (and whether it is wise?) to
> have a mode that will bypass this (otherwise very sensible) control?
>

Seems like a reasonable request. Doesn't make sense to refuse to start
up hidden services with a legitimate number of intro points. I added a
positive comment to #21033 which was courteously opened by David.

---

BTW and to slightly diverge the topic, I really like this experiment and
its blazing fast results, but I still get a weird feeling when I see
that to start functioning it requires 432 descriptors uploaded to the
HSDir system (72 daemons * 6 descriptors). To be clear, I think this is
fine for an innovative experiment like this, but it's not a setup that
everyone on the network should replicate. I guess to improve the
situation here, we would need some sort of "onionbalance complex mode"
where instead of uploading the intermediary descriptors to the HSDir
system, we upload them to an internal master onionbalance node which
does the intro point multiplexing.

Best of luck with your experiment :]