[tor-project] Sponsor 38 update (Shadow simulator)

[Jim Newsome]

Project management:

  *

    We’ve filled in a more granular plan in the _Sponsor 38_
    <https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor38>page.

We’ve made some progress on _Shadow_ 
<https://github.com/shadow/shadow>’s milestone _Improve development 
assurance and velocity_ <https://github.com/shadow/shadow/milestone/15>. 
Notably:

  *

    Fixed remaining compiler warnings in our continuous-integration
    environments, and enabled -Werror there. _[#726_
    <https://github.com/shadow/shadow/issues/711>]

  *

    Added continuous integration for shadow-plugin-tor, testing each
    pull request of shadow-plugin-tor against stable versions of shadow
    and Tor [_#85_
    <https://github.com/shadow/shadow-plugin-tor/pull/85>, _#86_
    <https://github.com/shadow/shadow-plugin-tor/pull/86>, _#90_
    <https://github.com/shadow/shadow-plugin-tor/pull/90>], and also run
    it for pull requests in the Shadow plugin against stable versions of
    shadow-plugin-tor and Tor [_#727_
    <https://github.com/shadow/shadow/pull/727>]

_v1.14.0_ <https://github.com/shadow/shadow/releases/tag/v1.14.0>of 
Shadow was released. In addition to the changes above, it modularizes 
the router queue management algorithm and makes CoDel the default algorithm.

Our current focus is on prototyping Phantom: a new architecture for 
Shadow that will run applications in their own processes rather than 
using a custom ELF loader to load them directly into Shadow’s process. 
The new architecture will work with unmodified program binaries (rather 
than having to recompile them with -fPIC), should be stabler and easier 
to maintain, and may have performance benefits.

Because Rob Jansen and Ryan Wails at NRL are actively involved in this 
development work, NRL regulations require that the active development 
happen in a private branch. When it’s ready to be upstreamed, they’ll go 
through an NRL release process and merge it into the public repo. In the 
meantime there is a place-holder issue [_#738_ 
<https://github.com/shadow/shadow/issues/738>] in the public repo.

Recent progress on Shadow-Phantom:

  *

    Added ability to simultaneously support alternative “thread”
    strategies for controlling and communicating with plugins.

  *

    Working proof-of-concept of “Shim-Pipe” threads, which use
    LD_PRELOAD to interpose the libc API to call a version of the
    “syscall” function that communicates via Shadow using a pipe.

  *

    Nearly done with a shared-memory-based IPC mechanism for the
    Shim-Pipe, which will be used for syscall pointer arguments.

  *

    Working proof-of-concept of “Ptrace” threads, which use ptrace to
    attach to plugin threads, and intercept and service syscalls. This
    approach is expected to be a bit simpler (the tracing process can
    directly read and write the tracee’s memory, making marshalling
    easier) and more reliable (e.g. handle direct usage of the syscall
    instruction), but be less performant (likely more context switches
    between Shadow, the OS, and the plugin).

In the coming month our main focus will continue to be on 
Shadow-Phantom. We intend to flesh out the functionality enough to start 
doing simple networking benchmarks, and if results look promising will 
continue to flesh it out enough to run Tor.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-project/attachments/20200319/a1fca8fb/attachment-0001.html>