[tor-project] [TSoP] stem.client - status report #6

[Dave Rolek]

Hi everyone!

This is my sixth status report for the stem.client TSoP project [1].
You may see my previous reports on the tor-project ml. [2][3][4][5][6]

This report is about a week late.


# Updates

It's been roughly 2.5 weeks since my last update.

I've still suffered from illness during this time - I'd say it knocked
out most of a week :( - but I think I've made some decent progress in
stem.client. I'm working through it, somewhat successfully.

## stem.client

I'm working through my plans for the stem.client roadmap.

You may recall from a previous status report [5] these early items on
the roadmap:

1. refactor: Relay cell encryption/decryption/processing
2. centralize ORPort reads/sends (demux/mux) at connection level
3. implement required RelayCell subclasses (e.g. parsing of decrypted
body)

(1) is mostly done in #27112 [7], with just a few more changes I wish to
make [8].
Both (2) and (3) have had some groundwork laid with #27112, as well.

Notably I've created an intermediate RawRelayCell class that can handle
arbitrary payloads (mostly to be used for encrypted payloads), and
started to move the finer details of encryption/decryption/processing
into the Cell abstraction.


## stem, general

Discovering a py3 bug in #26432 [9], I filed #27030 [10].

I concurrently ran into some of the problems Damian did, and took notes
about / fixed them concurrently:
* hashing super() causing an infinite loop, a la
  b48ce682ef1a5fc1d4237d106700ec929f49246b (but not exactly this) [11]
* a bit of cleanup from #26420 [12]

I ran into a lot of trouble with the tests in #27053 [13], and provided
a few rounds of input that has helped move that ticket forward.

## other project-related

I convened with plcp a bit over IRC in discussing stem.client code and
functionality of the tor-spec, reviewing some of their code [14].

And I did some stem ticket bookkeeping / responses.

## tor, general

I've continued to do some bookkeeping / ticket triage as I run across
things:
* #25573, #25574 "DropMark"
* #26228, respond on PR and close out PR
* probably more at such lower notability that I didn't take a note about
  it

## misc

I've improved my IRC client config for rejoining the registered IRC
channels on reconnect - I should be easier to reach now, even when I'm
not personally paying attention to the IRC client.


# What's next

There's only a bit over a week left in the project.
I plan to focus as much as possible on stem.client!

## stem.client

Please refer above to the roadmap bullets.

I've highlighted a few more changes I wish to make for (1) in the ticket
[8].

Between (2) and (3), I'm not sure what's best to tackle next.

For (2), the work primarily involves rearchitecting the recv/send into
its own thread(s) and centralizing the handling. I've done that sort of
work before, so I think I could design and implement that well.

For (3), the work is a bit more mundane, but it *does* provide the
building blocks to do higher-level management of circuits, like
extending a circuit (building blocks, naturally: EXTEND[ED][2] cells).

I'm thus leaning to tackle (3) first.

These are both important, but I think (3) enables other work to be done,
whereas (2) is an architectural improvement that improves stability and
usage, but doesn't open way for much functionality.

## tor-spec

File tickets to track spec improvements resulting from TSoP experience.
(These are bookkeeping / awareness tickets for my WIP changes.)

I still haven't filed these, and the spec has changed a bit, so I'm
deprioritizing this further until after stem.client coding.

## misc

I've generated a backlog of tasks that I haven't gotten to, including
ones I've mentioned in previous status reports. I'm deprioritizing these
for now, but I really do hope to get to them.

I'm not going to repeat / enumerate them here, for sanity's sake :)


# Other Tor things

Multiple in-person chats, yet again. While a sample size of 1 (i.e. me)
is by no means scientific, I'd say there's a definite, growing interest
in Tor and digital privacy overall.


# Closing

With illness persisting, I had another period where I stopped reading
IRC scrollback proactively, but I think I might be back on track again.

Nonetheless, please still mention my nick (dmr) if you want to get my
attention!

As with before, please don't hesitate to reach out to me via IRC or
email!


Thanks,
Dave


[1]
https://lists.torproject.org/pipermail/tor-dev/2018-April/013090.html
[2]
https://lists.torproject.org/pipermail/tor-project/2018-May/001811.html
[3]
https://lists.torproject.org/pipermail/tor-project/2018-June/001830.html
[4]
https://lists.torproject.org/pipermail/tor-project/2018-June/001858.html
[5]
https://lists.torproject.org/pipermail/tor-project/2018-July/001886.html
[6]
https://lists.torproject.org/pipermail/tor-project/2018-July/001917.html
[7] https://trac.torproject.org/projects/tor/ticket/27112
[8] https://trac.torproject.org/projects/tor/ticket/27112#comment:2
[9] https://trac.torproject.org/projects/tor/ticket/26432
[10] https://trac.torproject.org/projects/tor/ticket/27030
[11]
https://gitweb.torproject.org/stem.git/commit/?id=b48ce682ef1a5fc1d4237d106700ec929f49246b
[12]
https://gitweb.torproject.org/stem.git/commit/?id=d1284e6d0301de59ddf286b5e799dee50a5d4a5a
[13] https://trac.torproject.org/projects/tor/ticket/27053
[14]
https://github.com/plcp/stem-client/tree/6593f56f6dfdfc54102091b537b63f8dd1e855ed