[tor-dev] Multi-threading throughout

[Ian Goldberg]

On Wed, Jan 09, 2019 at 08:42:18PM +1100, Todd Hubers wrote:
> There are early plans to distribute crypto operations across multiple cores
> [https://trac.torproject.org/projects/tor/ticket/1749], but there might be
> a better way.
> 
> (I registered, but I couldn't find a way to annotate the ticket, so I'm
> emailing for now)
> 
> The ticket states the reason being to saturate the bandwidth available (by
> using all the cores as efficiently as possible).
> 
> I don't understand why a relay needs to have a "main thread". Network
> traffic arrives as an async operation and can be sent back out
> asynchronously. So a final strategy shouldn't have a central thread. The
> main thread might still be needed for startup, runtime adjustment, and
> system upkeep, but not for the core network-crypto processing; that should
> never need to touch the main thread.
> 
> The current proposal speaks about multi-threading crypto operations, let's
> call that "A) Speed - Speeding up processing of a single cell". Instead, I
> propose "B) Concurrency - Restructuring so multiple cells can be processed
> concurrently".
> 
> A cell of data should arrive via IO-Completion thread on a random CPU core,
> have crypto transformation applied on the same one core, then be dispatched
> onward out via the network. This seems to be quite a simple approach where
> I would think crypto code can remain the same "single-threaded"
> implementation.
> 
> Approach [A] will have diminishing returns as the number of cores
> increases. You can only break up a cell unit of work so much until you're
> encrypting one byte per cpu core. However, with approach [B], if you have
> millions of CPU cores (as an extreme) you can be processing millions of
> cells concurrently. Therefore, I believe approach [B] would be more
> scalable.
> 
> What do you think?

You'll have troubles if cells *on the same circuit* try to be processed
in parallel on different cores, at least with the current circuit-level
crypto.  But, once circuits are established, handing each circuit to a
different thread/core (or more clever worker structure) is something
that I think at least boradly makes sense, and indeed I have been
proposing to have my students work on.
-- 
Ian Goldberg
Professor and University Research Chair
Cheriton School of Computer Science
University of Waterloo