[tor-reports] dgoulet's November 2014

[David Goulet]

# Status report for November 2014

My contract started on November 1st with most of my time on SponsorR.

I went for hidden service code spelunking at the beginning of the month
to get a better understanding of this subsystem.

- Setup an experiment to profile the performance of an HS service. Basic
  IRC server and I fired one hundred client connections through
  different circuits and used "perf" profiling on the service side.

  Posted an update on #8902 for this with the results here:
  https://people.torproject.org/~dgoulet/tor-hs-perf-100-circ.png

  For the needed optimization of this work: #13738, #13739

- Help draft a proposal for collecting HS statistics with asn and
  karsten that has been split into a proposal made by asn/karsten to
  collect HS usage on the real Tor network. The other part will become a
  tech report soon (karsten work).

  Reference: #13509, #13792

- Applied Roger's patch for collecting some basic HS stats (#13195) on
  my fast relay to investiguate #13287 (Investigate mysterious 24-hour
  lump in hsdir desc fetches).

  Modified it a bit to log the hashed service descriptor that was being
  fetched over and over again thus giving us a possible answer:

  https://trac.torproject.org/projects/tor/ticket/13287#comment:7

- Went to Boston for a full week to collaborate with nickm on lots of
  areas of the Tor code base. Useful mentoring and guidance.

- I've started instrumenting the HS subsystem with "trace events" on
  which we can hook a user space tracer (like DTrace, SystempTap,
  LTTng). The goal is to have a tracing framework, *ONLY* for private
  network and should NEVER be deployed in production, that allows us to
  easily and efficiently gather low level information in order to build
  a performance baseline of the whole HS system.

  I've hooked LTTng user space tracer to the instrumentation. Work can
  be found here (#13802):

  https://gitweb.torproject.org/user/dgoulet/tor.git/shortlog/refs/heads/hs-lttng-025

  It's based on 0.2.5 and the reason is to be able to measure both 0.2.5
  and 0.2.6 for comparison.

- Setup a Chutney network that supports an HS service. I ran an
  experiment to measure HS system timings. For that, I used the above
  LTTng branch with an analysis script of the trace.

  Preliminary results on Introduction Point build time for HS service
  (for which I'll post a much more detailed report and documentation
  later on along with the tools needed).

  0.2.5 --> https://people.torproject.org/~dgoulet/ip.png
  0.2.6 --> https://people.torproject.org/~dgoulet/ip-026.png

- Ticket work:

  Performance analysis:
    #8902 - Rumors that hidden services have trouble scaling to 100
    concurrent connections
	#13664 - Potential issue with rend cache object when intro points
	falls to 0.
    #13739 - Optimize the functions called in
    circuit_launch_by_extend_info()
    #13790 - Refactor and add comments to new_route_len()

  Wrote patches:
    #13667 - Prevent port scanning of hidden services
    #13698 - Wrong failure report when closing parallel intro points

  Investiguation:
    #13287 - Investigate mysterious 24-hour lump in hsdir desc fetches
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 603 bytes
Desc: Digital signature
URL: <http://lists.torproject.org/pipermail/tor-reports/attachments/20141130/918faf71/attachment.sig>