[tor-dev] Hidden service performance

[David Goulet]

FYI, these are the stats we have now:

https://trac.torproject.org/projects/tor/ticket/13792#comment:4

Cheers!
David

On 17 Dec (20:20:50), David Goulet wrote:
> Hello tor-dev,
> 
> This email is to detail the work being done on a framework to do
> performance measurements for hidden service. I'm looking to get some
> feedback on the whole idea, improvement and help on possible analysis.
> If this work is not thrown out the window, I'll definitely do a Wiki
> page for a detail documentation but for now this is still a bit too
> early I think.
> 
> *** PLEASE NOTE THAT THIS FRAMEWORK IS _NOT_ INTENDED TO BE USED ON THE
> REAL TOR NETWORK, ONLY TEST NETWORK! *** (I'll even add a check to make
> sure TestingTorNetwork is set to 1 else tor should not run.)
> 
> The whole idea of this effort is to have a way to do very precise
> measurements of hidden service so we can come up with a performance
> baseline and have a reliable way to measure our progress on improving
> HS. We want to run this on large private network such as Shadow in order
> to get a better understanding of this whole HS subsystem. Finally, this
> can be extended to all sorts of component in tor in the future to help
> us analyze the beast that tor has become. :)
> 
> This work uses tracing[0] for measurements. For that, I've instrumented
> the Tor code base with "tracepoints" that allows us to hook a tracer to
> those so we can collect data and do analysis afterward.
> 
> The following branch contains a bunch of tracepoints mostly for HS and
> uses the LTTng[1] user space tracer. It's based on tor 0.2.5 version
> because we wanted to measure regression or improvement between 0.2.5 and
> upcoming 0.2.6 (especially with Andrea's new scheduler work).
> 
> 	https://gitweb.torproject.org/user/dgoulet/tor.git
> 	[branch: hs-lttng-025]
> 
> To compile this branch you need to install lttng-ust (tracer). It's
> pretty easy to install, you can find out how here[2]. (You will need
> version 2.4.0 or above).
> 
> 	./configure --enable-lttng-tracing
> 
> In "src/or/trace/events.h", you can find all tracepoints that have been
> added with the data they collect. It is a bit too tidious to explain
> them all to you in this email but every "tor_trace_*" line is a
> tracepoint that you can grep in the code to see their location thus have
> a clearer picture of what they record.
> 
> First, if you see any tracepoints that record something but are not at
> the right place in the code, you should definitely let me know. Second,
> telling me where tracepoint should be added also is very useful! No need
> to understand the full tracing system, just "that function should have a
> tracepoint to record this", I'll do the rest.
> 
> Now to use tracing with Tor and analyze the data, you can find useful
> information and examples in this repository. (Pretty sure it's missing
> some documentation so please point it out!)
> 
> 	https://gitorious.org/tor-hs-stats/tor-hs-stats
> 
> Here are examples of graph that we can do with the collected data from
> the traces.
> 
> 	(Introcuction Points Ready Time)
> 	https://people.torproject.org/~dgoulet/hs/ip-025.png
> 	or
> 	(Rendezvous Point Ready Time)
> 	https://people.torproject.org/~dgoulet/hs/rp-025.png
> 
> Now to make analysis, I've build what is called in the tracing world a
> "state system". See that as a way to represent different objects in an
> application that are populated by the data that has been collected with
> tracing (trace). For this, please take a look at "lttng-analyze.py" in
> the gitorious repository. That script parses the trace event by event
> (event is what a tracepoint generated), populates the state system which
> is analyzed after the trace is fully parsed. It's pretty straight
> forward and here is the power of the state system. Once build from a
> trace, we can run *ALL* sorts of analysis on it to measure timings, find
> bad behaviors, etc..
> 
> If that interests you, I encourage you to play with all this and see for
> yourself what can be achieved thus opening your eyes on the
> possibilities. :)
> 
> Ok, so there is a lot to explain to make things clear. I'm pretty sure
> I'm forgetting some here. Understanding the benifits of tracing is not
> that trivial at first glance so I'll be really happy to explain it to
> anyone asking.
> 
> Any feedback is positive feedback! I'm happy to answer any questions.
> I'll most probably follow up this with a post about some results at some
> point.
> 
> Cheers!
> David
> 
> [0] https://en.wikipedia.org/wiki/Tracing_%28software%29
> [1] https://lttng.org
> [2] https://lttng.org/download/



> _______________________________________________
> tor-dev mailing list
> tor-dev at lists.torproject.org
> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 603 bytes
Desc: Digital signature
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20141219/42ffb23f/attachment.sig>