[tor-scaling] Summary of 5/31 meeting; next steps

[Mike Perry]

David Goulet:
> On 31 May (20:39:00), Mike Perry wrote:
> 
> Thanks for this summary Mike!
> 
> [snip]
> 
>> In the meantime, with input from folks on this list and on the wiki
>> page, I would like to add the EWMA re-tuning experiment, fill out the
>> KIST tuning experiment, and flesh out the metrics section to highlight
>> metrics that need new data collection. (I will start separate threads
>> for this on-list as I run into questions -- I have several already).
> 
> (Maybe this could be on the thread you want to start?)

Maybe. There's still some experimentation process stuff here that is
directly related to the plan summary.. I'll leave it here for now unless
we have major plan changes that result from this discussion.

As soon as we start diving down into metrics and parameter values for
EWMA/Kist, we should make a new thread for that.

> Just an observation here. The real Tor network is very heterogeneous in terms
> of "tor versions". There is probably ~95% Linux and the rest BSDs but the
> point I want to make is about the different cell scheduler between relays.  It
> creates a big partition of KIST relays, KISTLite relays and non-KIST relays.
> (And any new improvements we will release will create more groups like that.)
> 
> We currently have ~1000 that are on 0.2.9, one of our LTS until Jan 2020.
> These relays are running the Vanilla scheduler versus the rest either the KIST
> or KISTLite scheduler.
>
> Thus, we get this mixture affecting performance on a circuit so tweaking EWMA
> parameters on such a setup (the Tor network ;) could not result in what we do
> expect from experimentation that is more likely to only have the same Tor
> version.

In the short term, I see this as an important thing to test our
simulators as being able to reproduce. I expect this version
heterogeneity to result in larger variance in performance than in a
homogeneous network. We'll likely see different amounts of improvement
per torperf fetch, depending on the number of 0.2.9 relays in the path
of that fetch. This will result in a slower-climb CDF (more variance)
than if all relays experienced improvement (cliff-like CDF; less variance).

We should see the same thing in any network simulators that we test
with. Any that are unable to accurately run heterogeneous Tor networks
will fail to match these results from the live network, and show us that
we can't accurately predict network effects with them. (This is the
whole point of doing this tuning on the live network, and then
back-testing on our favorite simulators).

But this is a good point for long term: LTS is really going to make it
impossible to substantially improve the Tor network with any relay-side
new development improvements prior to Feb 1, 2022 (when the 0.3.5 LTS
expires).

Most funders are unlikely to be impressed with our inability to provide
big-win results sooner than 3 years from now, especially wrt our
worst-case performance/variance problem. That's probably longer than it
would take just to build a new Tor-like network, instead... We need to
find a better way to support Debian/stable relay operators, other than
simply letting them hold back the network and increase development time
and maintenance costs by running ancient software.

> And I would also like to point out that for optimal performance, whatever
> client measuring it should follow the workaround in #29427 (bring down the
> KISTSchedRunInterval to 2msec.

How certain are we about this value? And what about relays?

I wanted to tune this value as a separate experiment:
https://trac.torproject.org/projects/tor/wiki/org/roadmaps/CoreTor/PerformanceExperiments#KIST

Does that make sense? Can you help provide info for the fields there,
and clarify if we want to set different values for clients vs relays?

I also assume this means that the KIST tuning must be done before the
EWMA tuning?

-- 
Mike Perry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://lists.torproject.org/pipermail/tor-scaling/attachments/20190603/a26b4ef9/attachment.sig>