[tor-dev] txtorcon: fixing APIs, painting bikesheds

[meejah]

There are several rough edges and "regrettable" API choices in txtorcon
that I'd like to fix as a "1.x" release. I also had several productive
discssions with Brian Warner (thanks a lot!) suggesting some things to
make using txtorcon easier.

To that end, I am collecting some of these thoughts as well as massively
re-organizing (re-doing?) the documentation in a branch called
"release-1.x". ReadTheDocs is building this now, too. Some highlights of
what I'm doing or find annoying currently:

 - Move a bunch of things to "private" attrs/functions that were never
   supposed to be public (i.e. prefix a bunch more things with "_")

 - Eliminate need to deal with the "bootstrap" Deferreds (via factory
   functions and/or other help methods) (mostly this is true already)

 - Python3 (works in release-1.x branch now, minus txsocksx)

 - Re-org docs, add "programming guide" (instead of walkthrough,
   tutorial and HOWTO)

 - consolidate examples (there's too many, and some are confusing)

 - introduce a "Tor" object that abstracts "a running tor instance"
   (whether launched by txtorcon or just connected-to) as a Builder
   or Factory for other useful objects. This would be "the" high-level
   API that most users would interact with. Something like:

       @inlineCallbacks
       def main(reactor):
           tor = yield txtorcon.launch(reactor, ...)  # or .connect()
           torstate = yield tor.create_state()
           print("all circuits:", torstate.circuits.values())
           endpoint = tor.create_client_endpoint('torproject.org', 443)
           onion = tor.create_onion_endpoint(port=80, private_key=None)
           # etc.

   This would get rid of "launch_tor", "build_local_tor_connection" and
   others, giving just "launch()" or "connect()". get_global_tor() would
   also return a "Tor" instance. Also, launch() would *not* take an
   arbitrary TorConfig (as launch_tor does now) but instead would expose
   a very few critical options as kwargs (and if you need to mess around
   with obscure options, you'd do so after launch).

 - .get_info() should never have returned a dict (instead: list of
   results, in same order as keys)

 - consolidate all the various types of "onion/hidden-service" things to
   support ephemeral and "on disk" services, as well as
   authenticated. There are currently conceptually 6 variants from tor:

       ephemeral: plain, basic-auth (not in tor), stealth-auth (not in tor)
         on-disk: plain, basic-auth, stealth-auth

   This is further complicated by the fact that both the authenticated
   services are really, logically, a "list of clients" each of which has
   a name, a secret (and for stealth, a separate .onion address). My
   current design *ideas* are in the release-1.x branch, but boil down
   to interfaces: IOnionClients (for basic + stealth auth) which is a
   collection of IOnionClient instances (has secret + name, and is a
   subclass of IOnionService) and IOnionService for all the rest (has
   hostname, ports, private-key). This stuff is the most in-flux still.

   "create a new onion service" would be available at the high level via
   one or several method-calls on Tor() instances which would return new
   concrete IStreamClientEndpoint or IStreamServerEndpoint implementers

 - For "I want a Stream over Circuit X", add a ".stream_to(endpoint)"
   method on Circuit instances that gives you a Twisted
   IStreamClientEndpoint implementation causing the stream resulting
   from .connect() to be attached to that specific circuit.

 - Introduce a configurable "CircuitBuilder" to more-easily control
   construction of "types of" circuits (instead of just "build a circuit
   through these exact relays" as .build_circuit() provides). For more,
   see: https://github.com/meejah/txtorcon/issues/125

 - re-consider all the magic (attribute-style access) of TorConfig

 - Add a "composible"-style API for listening to Circuit and Stream
   events (instead of having to implement ICircuitListener etc).

I would be happy to hear of (other) existing APIs that are awkward or
confusing as well as opinions on the general colour of my
bikeshed. Please beware that this branch is very much "in progress" ;)

As far as current users of 0.x releases: I will not break these, and
will backport any critical fixes to them. My preference would be to
simply work with existing users (e.g. via pull-requests to your repos)
to let them seamlessly upgrade to 1.x as soon as 1.0.0 is pushed out the
door. Where possible I can provide wrappers for backwards-compatiblity
for any removed/re-named methods

My goal here is *not* to just gratuitously break things "because
aesthetics" but to make the "API surface" of txtorcon smaller, easier to
understand and better documented.

How to help:

 - look at http://txtorcon.readthedocs.org/en/release-1.x/ (note that
   some of the docs are lies as the code doesn't always exist yet, or
   worse yet exists but isn't what I want)

 - express opinions (here, #tor-dev, via any of the methods at
   https://meejah.ca/contact)

 - *maaaybe* look at the new code (but I'm trying to write the docs
   first, so those are the most-useful to look at). I want the docs to
   reflect the upcoming reality, and will change the code to match.

Thanks,
meejah