[tor-dev] The Shadow Simulator will soon use cooperative threading to better support plug-ins

[Rob Jansen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hello Shadow! Hello Tor!

For the last couple of weeks, I have been overhauling the way Shadow
interacts with plug-ins, and I want to make sure you (and others
developing/using Shadow plug-ins) are aware of the upcoming changes.

I am enhancing Shadow to support the following two capabilities:
   1. Run plug-ins that make blocking system calls
   2. Run plug-ins that create and use pthreads

To achieve this, I am using a user-space cooperative threading library
called pth:
http://www.gnu.org/software/pth/

Pth works by storing and swapping stack states using makecontext and
swapcontext (see man pages), so that we can intelligently switch back
to Shadow's execution stack when the plug-in makes blocking libc
function calls, and switch back to the plug-in stack only when its
file descriptors are waiting for I/O or timers have expired. This same
technique allows us to support would-be plug-in pthreads as
application layer pth threads instead, which will help us avoid
expensive kernel context switches. This will be especially
useful/efficient when running thousands of nodes simultaneously, which
normally would consume thousands of pthreads which are not Shadow aware.

I have modified the pth library to be reentrant and thread-safe using
caller-supplied states and thread-specific data, which I needed
because Shadow itself runs with multiple simulator worker threads.

Cool, huh!? So what does this mean?

With this fancy new approach, we must still build each plug-in using
LLVM/Clang in order to make use of Shadow's cool state-swapping trick,
but we may no longer need to do other customization, particularly
those historically needed for programs that block or use threads.
Also, with these changes, Shadow's "API" has been removed, and Shadow
now just calls the "main" function symbol in each plug-in directly,
and the plug-in need not know anything about Shadow.

I'm excited about these changes, and hope they will improve support
for new plug-ins while also reducing maintenance in the long term. So
far I have Shadow's built-in traffic generator running correctly with
the new approach, and am now working on Tor. None of this code is
public yet, but if you are interested in an example of how this would
work or want some early access to the code, drop me a message off-list.

Cheers!
Rob

- -- 
Rob Jansen, Ph.D.
Computer Scientist
Center for High Assurance Computer Systems
U.S. Naval Research Laboratory
rob.g.jansen at nrl.navy.mil
robgjansen.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCgAGBQJVu4lYAAoJEO3Z5w0UVGXoASAP/3qjW0551xxnrLwWQcrXhbiN
cf7hryzU7vDzKRkdzzNbEive0JFZhDONbruJ5Lcx1BW4/uvFZ1emW0FuZFri1m7b
xWW6Pp43cMDGeIaMzmWZ9UAea59wirFk/C47R/jDBkXWnZpz5Arc1DjryPbj2LqN
oPcfd9JDyE9zk2Y0sSytYt3IulKA62ZOV7+WbIwhHUTj70OgXFvVLhLMMVgOBEDO
rHWxDg/+5mC8WqEg4L7lS8/ivAjr1YboA4PV0uSPzgZ+208iQBbsF3DoCG/8D9kY
83xpX/+3aGdWl7XjrncpA5x7gt392fX/3EL9k8lUYu2PKPaOWmJ0/xWbGoXGJOhA
epAWPr95X7DFzpcsS59sq6usDeiltFllzfxgA5TbDsUgVwOa9227ygYtK/MVifi6
gYkpZzC6YEMDR6id0ZOwwqYKTsx1Sxomyz30w5iAp1l+Grb10bhUZlfqsNx9PkuI
HdBMfvNziApSOQBge36QwskGFlJobfGdpL5w0yep1MSmBJCLVSCopCllLVcbDzQX
ipbjmHjJXHNaIL7DHm073VJe2XAjcoMPHHs/nhq0vTgrMFGMDsNvMJ7+66xz3iwZ
V3WUjEP0fPjgGZ8MTWj2HjGOQyENa5yAmAiBJUosg1fzgeinushWfE+Zm4m0sVEJ
BJttAL/8SnI4DpbIqq6e
=c1wf
-----END PGP SIGNATURE-----