[tor-dev] txtorcon 0.2

[Damian Johnson]

> I was thinking about this a little last week -- it would certainly be
> nice to abstract more of the "general parsing stuff".

We plan to put most of the parsing stuff into the stem.response
module, which txtorcon could use without touching any of the threading
stuff. For example...

https://gitweb.torproject.org/stem.git/blob/HEAD:/stem/response/protocolinfo.py

There is not much there at present since I've been focused on other
areas. Ravi's GSoC project is to make this and our Controller class
cover the complete control spec.

One minor gotcha that I should sort out, if you want to use stem, is
for the ControlMessage class to have a simple string constructor
since, at present, it expects to get partly parsed content from the
socket...

https://gitweb.torproject.org/stem.git/blob/HEAD:/stem/response/__init__.py

> There are a few
> gotchas since the threaded versus event-based way to get information
> from the protocol is pretty different.

It is, but this just means that we'd have trouble sharing the
ControlSocket and BaseController subclasses. That's a fair bit
(including the stem.connection module), but still plenty that could be
shared.

> I've tried to imagine a threaded-friendly wrapper around at least
> txtorcon.TorControlProtocol which might not be hard for the simple
> command-response things (but see below).

Great. If we could come up with a common interface for simply sending
a message and receiving its reply then the stem.connection module
would be no trouble.

> *Ideally*, such a library could leverage the parsing code in Tor
> itself -- if at least the "utility" methods in Tor could be published
> as a shared library, a "ctypes" wrapper could easily be made with a
> more-Pythonic interface around that.

Eek! I'd hate to have the distribution pains that any C component
would bring. If Tor does not conform with its spec and allow for
independent implementations in other languages then that's a Tor bug,
and testing that Tor conforms to its own spec is part of the goals for
the integ tests that I'm writing.

> Another thing I think would be really nice is to be able to get
> grouping and documentation information about config options from Tor

Do you mean if a particular config option belongs to Relays, Hidden
Services, etc? If so then this might interest you...

https://gitweb.torproject.org/arm.git/blob/HEAD:/src/resources/torConfigDesc.txt

Arm parses the tor man page when it starts up into files like these so
it can present the category of configuration options, and help text.
Here's the parsing code...

https://gitweb.torproject.org/arm.git/blob/HEAD:/src/util/torConfig.py#l97

I plan to clean it up and move those capabilities to stem, but that'll
be a while.

> I'm mostly-away until around the 18th, but perhaps we could meet on
> #tor-dev after that and discuss further?

Certainly.

> Are there specific things
> besides descriptors that you think could be easily abstracted out of
> stem (and/or useful for txtorcon)?

Pretty much anything outside of stem.socket, stem.control, and maybe
stem.connection (depending on the common interface discussion above).

Cheers! -Damian