[tor-dev] Using Shadow to increase agility of Sponsor R deliverables

[David Goulet]

On 20 Nov (14:45:12), Rob Jansen wrote:
> Hi David, Roger,

Hello!

> 
> I think it would be great if we could show off some HS performance improvements at the next Sponsor R PI meeting in January, both as a way to show that we are making progress on the performance front and also to help demonstrate our agility and how quickly we can get results on new designs. I’m here to advocate the use of Shadow to help in this regard.
> 
> So far, the list of deliverable that we could possibly show improvements is quite small:
> https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorR#Tickets
> 
> From the list of completed tickets, the only one that stands out to me as providing a performance boost is #13211 (Allow optimistic data on connections to hidden services):
> https://trac.torproject.org/projects/tor/ticket/13211
> 
> This seems like a somewhat small change; despite that, do we think this may be something worth simulating to verify it works as expected and understand the extent to which it reduces time to first byte for HS downloads?
> 
> Are there other HS performance improvements that we think may be ready by January?

Here is what we (Karsten, George and me) are up to on that front for
January. (Please guys, feel free to fill the blank if I am missing
anything).

We are aware of the january deadline so we've split up the work in two
parts. George/Karsten are working on a proposal to gather HS statistics
on the real Tor network to answer some questions that are presented
here.

https://people.torproject.org/~karsten/volatile/238-hs-relay-stats-2014-11-20.txt

On my part, I have a chutney network with an HS and clients that fetch
data on it. I'm currently working on instrumenting the HS subsystem so
we can gather performance data and analyze it for meaningful pointers on
where are the contention points, confirm expected behaviors, etc...
I'll begin soon updating the following ticket with more information on
the work I'm doing. (I'm in Boston right now collaborating with Nick for
the week so things are a bit more slow on this front until monday).

https://trac.torproject.org/projects/tor/ticket/13792

This could be used also with shadow I presume. Since the deadline is
near us, I choose chutney for simplicity reasons here. I'll have a talk
with Nick tomorrow on how we can possibly have this instrumentation
upstream (either logs, controller event or/and tracing).

Please note that Karsten is helping everyone here for both parts! :).

On the host side of HS (meaning the HS relay itself), we have profiled
an HS service that is being hammered with hundreds of connections. The
client has been fixed to handle that also btw (#13698). I've talked this
one out with Nick also and we have an idea for a solution that is in the
mention in the ticket. Fixing that would in theory improve quite a bit
the host side performance of HS. You can find the info here.

https://trac.torproject.org/projects/tor/ticket/13739

Things are going forward, we still have some work ahead to gather the HS
performance baseline and start trying to improve it. I'm fairly
confident that the performance statistics in a private network will give
us a good insight on the current situation.

Feel free to propose anything that could be useful to make this thing
more efficient/faster/useful :).

Cheers!
David

> 
> Best,
> Rob
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 603 bytes
Desc: Digital signature
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20141120/f967d630/attachment-0001.sig>