[tor-dev] Scalability or Onionbalance for v3 ephemeral/ADD_ONION services

[Holmes Wilson]

Hi George,

Sorry for the slow reply here! Just getting back to this. 

>> For our application (a messaging app) it would be super useful to get the
>> full list of known online (or recently seen online) onion addresses in
>> possession of some frontend key. This would let us use onionbalance for
>> peer discovery instead of blindly trying the set of all known peers, which
>> won't work well for large groups / large numbers of peers.
>> 
> 
> Hmm, can you please give us some more details on what you are looking
> for? What is peer discovery in the above context, and what do you mean
> with "full list of ... onion addresses in possession of some frontend
> key"? I'm asking because the frontend key of onionbalance is also the
> onion address that users should access.

Our context is we are building a Discord-like team messaging app where peers are connected to each other over Tor, via onion addresses, rather than to a central server. So each user connects to a few peers, and messages travel across peers on a gossip network, and there’s a mechanism for syncing messages you missed, say, if you went offline for a bit. 

One problem we have is, when a new peer comes online, how do they know which other peers are online? Right now, they can try all of the peers they know about, or perhaps try recently-seen peers. But if there are hundreds of peers and only a few are currently online, it will be necessary to try many unreachable peers before finding one who’s online. So that’s not ideal.

One solution to this would be for each online peer to host the same onion service, using a shared key, in addition to their normal peer onion address. And at this address they could return a list of peers they knew were online. So a user would just have to connect to one address, at which point the Tor network would connect them to some online peer, and then that peer could tell them about other online peers. The problem with this approach, as pointed out by folks on this list, was that all those peers would have to really trust each other, since any one of them could go rogue and host malicious information instead of the peer list, gumming up the works. I’m not sure this is a fatal problem, since it would still *help* in cases where there wasn’t a malicious peer, and users could still fall back to the slower method of trying every peer. 

But what I’m wondering is whether there is any mechanism for a bunch of onion addresses that *don’t* completely trust each other to share a “meta” onion address on the Tor network, such that when the user looks up that identifier instead of getting connected directly to whatever content one of those onion addresses is serving, they get a list of all onion addresses that hold the keys to the “meta” address. 

It’d be like asking Tor, "show me a list of all onion addresses that have registered this meta address.” Sort of like asking, “show me a list of mirrors for this address…” at which point the user could try connecting to one or more of them, but would not have as serious problem if one of the sites went rogue and started serving useless content.

This is a bit of a long explanation, and my guess is that there isn’t anything like this and that the above scenario isn’t common enough to be worth targeting, but I was curious if anything like this had ever been discussed.

Thanks!
Holmes

> 
> Cheers!
> 
> 
>> I'd be interested in working with others on a spec for this!
>> 
>> On Mon, Jun 14, 2021 at 6:25 AM George Kadianakis <desnacked at riseup.net>
>> wrote:
>> 
>>> Chad Retz <chad.retz at gmail.com> writes:
>>> 
>>>> A quick glance at the code shows that ADD_ONION (i.e. "ephemeral"
>>>> onion services) doesn't support setting an Onionbalance
>>>> frontend/master onion address (specifically
>>>> https://gitlab.torproject.org/tpo/core/tor/-/issues/32709 doesn't seem
>>>> to have a control-side analogue). Would a feature request for adding a
>>>> `*(SP "OnionbalanceMasterKey=" OBKey)` (or "OBMasterKey" or whatever)
>>>> to ADD_ONION be reasonable? If so, just add in Gitlab?
>>>> 
>>> 
>>> Hell Ched,
>>> 
>>> that's indeed something that is missing and a reasonable feature
>>> request. A spec/code patch would be particularly welcome ;)
>>> 
>>>> Also curious alternative scalability and load balancing options for
>>>> ephemeral v3 onion services. I have read
>>>> 
>>> https://www.benthamsgaze.org/wp-content/uploads/2015/11/sucu-torscaling.pdf
>>>> but unsure if anything more recent has been written. Beyond that and
>>>> Onionbalance, any other interesting approaches I could employ
>>>> (assuming I can dev anything from a control port pov, but am wanting
>>>> to work w/ an unmodified Tor binary)?
>>>> 
>>> 
>>> Another complementary approach is to split the 'introduction' and
>>> 'rendezvous' functionalities to different hosts:
>>> 
>>> https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/255-hs-load-balancing.txt
>>> However it hasn't been implemented yet...
>>> 
>>> Cheers!
>>> _______________________________________________
>>> tor-dev mailing list
>>> tor-dev at lists.torproject.org
>>> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev
>>> 
>> _______________________________________________
>> tor-dev mailing list
>> tor-dev at lists.torproject.org <mailto:tor-dev at lists.torproject.org>
>> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20210723/5e8a6a53/attachment.htm>