[tor-dev] RFC: Trunnel -- a binary parser generator for Tor and more

[Nick Mathewson]

Hi!

Back when the Heartbleed vulnerability went public, I thought to
myself: "If we keep generating writing C code to parse binary formats
by hand, the same thing will happen to us sooner or later."

So on and off, I've been working on a little tool to automatically
generate binary encoding and parsing code based on C-like structure
descriptions.  It's a bit like nail[0], but more lightweight: it
doesn't support backtracking, sub-byte alignment, or any of several
other features that Tor doesn't actually want.

You can see a pre-alpha version of the tool (called "Trunnel") in one
of my git repositories [1].  The "doc" subdirectory has a simple
manual [2].

To make sure that it would suit our needs, I wrote up trunnel
specification files for the current Tor protocols, for SOCKS, and for
the new protocols in proposals 220, 224, and 229.  It seems to work
okay.

In case you want to see what the tool does without running it
yourself, you can find the SOCKS+proposal229 specification here [3],
and compare it with the generated header [4] and C implementation [5].

Trunnel is written in Python; my tests have 99% line coverage on the
Python code. The C code which is generated by those tests itself has
99% line coverage from other unit tests, so I'm thinking that it's
reasonably tested.  (Branch coverage is lower, but still pretty high.)

So, please check this out and have a look at it!  My open questions are:

  * Is this a good idea?
  * Is it a good idea to use this in Tor?
  * Are there any tricky bugs left in the generated code?
  * What am I forgetting to think of?

FWIW, my current plan is not to do a blanket replacement of our
existing parse code with Trunnel (yet) but rather to use Trunnel to
generate parsing code for new cell formats that are getting added in
0.2.6 and later.

[0] https://github.com/jbangert/nail
[1]  https://gitweb.torproject.org/user/nickm/trunnel.git .
[2] Also online at http://www.wangafu.net/~nickm/trunnel-manual.html
[3] https://gitweb.torproject.org/user/nickm/trunnel.git/blob/HEAD:/examples/socks5.trunnel
[4] http://www.wangafu.net/~nickm/volatile/socks5.h
[5] http://www.wangafu.net/~nickm/volatile/socks5.c

cheers,
-- 
Nick Mathewson