[tor-dev] Arti Report 3: July 21 through August 4

[Nick Mathewson]

Still quiet here for the last couple of weeks.  I've been moving ahead on
circuit timeout inference, and reviewing _lots_ of volunteer-submitted code
(and job applications).

The quiet part is coming to an end: Two of the engineers who have been on
vacation or leave are back, and we're all trying to ramp up.  We're also
hoping to be scheduling interviews with prospective co-workers soon, so
that's taking up some of our time.. but once it's done, it should accelerate
our pace a bit.

## Circuit timeout inference logic

I had hoped to be done with this by now, but it turns out that this code
touches on several key architectural issues that we'll want to make sure we
get right in the future.  I'll touch on them a bit here.

### Persistent state

The Tor protocol requires clients to keep persistent state across
invocations in order to achieve good performance and privacy.  Two
notable cases of this are in the circuit management code: client need to
remember circuit timeout data, and need to remember guard selection
data.

The C Tor client records these items in the same ad-hoc format used by
Tor's configuration code.  But for Rust, using serde seems like a better
fit.

I want to maintain the feature that Arti has now where you can run
multiple processes with the same state directory.  That could get
tricky in the long run, but I think it's a design feature we need if
we're going to keep Arti flexible enough for general use.

My first implementation here is going to be bare-bones, but I'm trying
to design it to grow and get replaced down the line.

### Inter-module event notification

Frequently one module needs to notify another when some event occurs.
In the circuit-timeout case, we need to notify the circuit manager
whenever the network parameters have changed.

We don't want to have the circuit manager probing the directory manager,
since that's a higher-level module.  And we don't want it probing the
`NetDir` object it gets whenever it's time to build a circuit, since
that would be a weird side-effect.

The best I've been able to come up with for now is to allow the
directory manager to expose a `Stream` of events for directory changes,
and to have the `TorClient` object use events from that stream to alert
the circuit manager to possibly changed network parameters.  I think
this should scale to other kinds of inter-module events in the future,
though I do have a slight concern that it makes `TorClient` harder to
reproduce with lower-level crates than it would be otherwise.  (There's
probably no avoiding that at this stage, though.)

### Non-user-initiated background tasks

The circuit timeout code sometimes needs to build testing circuits in
order to gather data, and the guard code does too.  If we code this in a
sloppy way, we'll wind up with three separate systems that all build circuits
for different reasons at different times.  With any luck, we can summarize
the triggers from all of these circuit types into a single system.

## Coming up

We've been working on refinements to the API in `arti-tor-client` to try to
simplify the API as much as possible while still exposing the full back-end
functionality.  Everybody should expect a lot of unstability here.

I am still hoping very much that the next time I write one of these reports,
we'll be done with circuit timeout inference, and I can talk about API work
and early progress on guards! :)