[tor-dev] Arti development report: July 7 through July 21

[Nick Mathewson]

# Arti Report 2: July 7 through July 21

It's been a quiet couple of weeks.  Most of the team has been on
vacation or leave for the last two weeks, so the Arti hacking has
been down to me (nick) for this period.

Our last report went out on the same day that our [announcement]
went public, and since then we've had an influx of attention and
volunteers interest.  Thanks to everybody who's been trying out the
code!  Special thanks to `trinity-1686a` for a
[stream-isolation] implementation, `lyuyuan` for
work on [RESOLVE and RESOLVE_PTR][resolve], and everybody else who's
been submitting patches!

## Circuit timeout inference

My own work for this period has focused on [circuit timeout
inference][cti].  (I decided to start with this because it's part of the
Tor protocol that I've worked with the least in the past.)

The approach is kind of complicated: in order to adjust our timeout, we
need to observe how long circuits actually take to
complete... including the circuits that we don't use because they
take longer than our current timeout timeout. So we have to keep
building circuits for a while even after they time out, so that we
can observe how long it takes them to finish.  Then we use an
estimator based on a Pareto distribution, and use that to determine
our actual cutoffs.

The core logic is now [implemented in Arti][pareto-code], and we
just have a few more pieces ([persistence][ticket-143],
[directory notification][ticket-144], and
[testing circuits][ticket-145]) to cover before we can call it done.

## Open problems

Our timeout tests rely on using a simulated view of time.  (Nobody
wants to wait 10 seconds for a simulated 10-second timeout!)  For
that, we have [testing code][rt-mock-time] to override the
asynchronous runtime's view of the passage of time.

But we have a bug somewhere that makes these tests unreliable.  The
[WaitFor] future, which is supposed to run a another future to
completion while advancing the current (simulated) time, doesn't
seem to be reliable.  As implemented, it advances the current time
even when the future it's testing _isn't_ blocked on time, and
sometimes advances time much too far while the tested future makes
no progress.  I've kludged it up by making it yield a lot and
advance time in tiny increments, but that's just a workaround.

Does anybody like [debugging Futures code][ticket-149]?

## Next steps

In the next two weeks I hope we can wrap up the outstanding issues above
in circuit timeouts, including the ones that require some
architectural decisions.

I'm also hoping that we can make progress on [guards]: that's the
last remaining really big security feature we have to do for
our 0.0.1 milestone.  Once that's done, we can move on to polishing
and infrastructure issues for the rest of the time we have for this
milestone.

Finally, we're going to be expanding our team!  Please have a look
at our [job-opening], and share it with anybody you know who is
interested in Rust and Tor, and who might be interested in a job.

Next update in two weeks!

[announcement]: https://blog.torproject.org/announcing-arti
[cti]: https://gitlab.torproject.org/tpo/core/torspec/-/blob/master/path-spec.txt#L364
[cti]: https://gitlab.torproject.org/tpo/core/torspec/-/blob/master/guard-spec.txt
[job-opening]: https://www.torproject.org/about/jobs/rust-dev/
[pareto-code]: https://gitlab.torproject.org/tpo/core/arti/-/blob/main/tor-circmgr/src/timeouts/pareto.rs
[resolve]: https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/39
[rt-mock-time]:
https://gitlab.torproject.org/tpo/core/arti/-/blob/main/tor-rtmock/src/time.rs
[stream-isolation]:
https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/38
[ticket-143]: https://gitlab.torproject.org/tpo/core/arti/-/issues/143
[ticket-144]: https://gitlab.torproject.org/tpo/core/arti/-/issues/144
[ticket-145]: https://gitlab.torproject.org/tpo/core/arti/-/issues/145
[ticket-149]: https://gitlab.torproject.org/tpo/core/arti/-/issues/149
[WaitFor]: https://gitlab.torproject.org/tpo/core/arti/-/blob/main/tor-rtmock/src/sleep_runtime.rs#L123