[tor-dev] Deployability of Python software.

[Arturo Filastò]

We were discussing last night with George about deployability of python
application on multiple platforms. 

In particular how it would work out if there were to be a python port of obfsproxy
and we wanted to have it deployed inside of the Tor Browser bundle.

The issues that he said were raised in other discussions with Nick and Roger
are mainly the following:

- How do we get a good Windows binary of the Software?
- How do we keep the size down to an acceptable level?
- What kind of performance drawbacks would we be experiencing?
- Is it even secure to do crypto in python?

I will try and address these issues as they are something that I ran into
also while designing AWAF (Anonymous Web Application Framework):
http://wiki.globaleaks.org/index.php/Awaf and 
https://piratenpad.de/p/AnonymousWebApplicationFramework

For packaging python software on Windows and OSX, what is generally done is
shipped a precompiled python interpreter and bundle everything up with a nice bow.

This technique is already quite tested in real world applications: an example that I
particularly like is Tucan Manager (http://www.tucaneando.com/development.html).

This application is basically a download manager written in python and gtk. 
The final size of the packaged software is 20MB. If you remove gtk this size goes down
to around 10MB.

What they are using to bundle up the application for Windows is py2exe and py2app for
OSX.

Another very widely used solution for packaging python applications in PyInstaller and
that is probably the solution I would recommend. Quite a few open source software
uses it already:
http://www.pyinstaller.org/wiki/ProjectsUsingPyInstaller

George also mentioned to me pypy, though I don't think pypy is ready for building shippable
application just yet.

The issue of size is something that we should come to an agreement on what is acceptable.
What is the maximum size that we are comfortable with shipping? We are already shipping
a TBB that has 25 MB of QT libraries in it, I don't think a 13 MB Python interpreter is going to
be killer.

With respect to performance I don't think it is particularly an issue. Python is pretty fast and if
it is not fast enough for what needs to be done you can always rewrite the code in C and
integrate that piece of application logic as a python binding.

By talking to some of the core python developers my understanding is that there is a way of 
securely storing keys in memory and wiping that memory region in python. It involves using
bytearray. We you override a cell in a byte array you are not simply dereferencing the pointer
to the python struct, you are actually overwriting that portion of memory.
I think I might write a blog post about this and illustrate what other python crypto software is
using to solve this problem (PyCrypto etc.).

In conclusion having a python interpreter shipped as part of Tor would allow developers of
anonymity related software to integrate their "Tor add-ons" into a Tor bundle easily. I am thinking
of for example making a Tor IRCD bundle, a Tor HTTPD bundle, etc.

What do you think?

- Art.