[tor-dev] RFC: Trunnel -- a binary parser generator for Tor and more

[Nick Mathewson]

An update:  Trunnel is now at version 1.3.  Its official repository is now
https://gitweb.torproject.org/trunnel.git .  Some code that  it has
generated has been merged into the Tor master branch for the 0.2.6
release series, though that code is not yet in active use.

  More review and comments are always appreciated. :)

On Wed, Aug 13, 2014 at 3:37 PM, Nick Mathewson <nickm at torproject.org> wrote:
> Hi!
>
> Back when the Heartbleed vulnerability went public, I thought to
> myself: "If we keep generating writing C code to parse binary formats
> by hand, the same thing will happen to us sooner or later."
>
> So on and off, I've been working on a little tool to automatically
> generate binary encoding and parsing code based on C-like structure
> descriptions.  It's a bit like nail[0], but more lightweight: it
> doesn't support backtracking, sub-byte alignment, or any of several
> other features that Tor doesn't actually want.
>
> You can see a pre-alpha version of the tool (called "Trunnel") in one
> of my git repositories [1].  The "doc" subdirectory has a simple
> manual [2].
>
> To make sure that it would suit our needs, I wrote up trunnel
> specification files for the current Tor protocols, for SOCKS, and for
> the new protocols in proposals 220, 224, and 229.  It seems to work
> okay.
>
> In case you want to see what the tool does without running it
> yourself, you can find the SOCKS+proposal229 specification here [3],
> and compare it with the generated header [4] and C implementation [5].
>
> Trunnel is written in Python; my tests have 99% line coverage on the
> Python code. The C code which is generated by those tests itself has
> 99% line coverage from other unit tests, so I'm thinking that it's
> reasonably tested.  (Branch coverage is lower, but still pretty high.)
>
> So, please check this out and have a look at it!  My open questions are:
>
>   * Is this a good idea?
>   * Is it a good idea to use this in Tor?
>   * Are there any tricky bugs left in the generated code?
>   * What am I forgetting to think of?
>
> FWIW, my current plan is not to do a blanket replacement of our
> existing parse code with Trunnel (yet) but rather to use Trunnel to
> generate parsing code for new cell formats that are getting added in
> 0.2.6 and later.
>
> [0] https://github.com/jbangert/nail
> [1]  https://gitweb.torproject.org/user/nickm/trunnel.git .
> [2] Also online at http://www.wangafu.net/~nickm/trunnel-manual.html
> [3] https://gitweb.torproject.org/user/nickm/trunnel.git/blob/HEAD:/examples/socks5.trunnel
> [4] http://www.wangafu.net/~nickm/volatile/socks5.h
> [5] http://www.wangafu.net/~nickm/volatile/socks5.c
>
> cheers,
> --
> Nick Mathewson