New passive performance metrics in Tor

[Karsten Loesing]

On 8/17/10 8:43 PM, Robert Hogan wrote:
> On Tuesday 17 August 2010 12:52:35 Karsten Loesing wrote:
>> Hi Robert,
>>
>> On 8/16/10 8:09 PM, Robert Hogan wrote:
>>> On Monday 12 July 2010 12:17:47 Karsten Loesing wrote:
>>>> Hi everyone,
>>>>
>>>> I'm planning to add new passive performance metrics to Tor so that we
>>>> can better understand why it's slow and how we can improve it. Here
>>>> is a list of performance metrics we already have and a few ideas for
>>>> new metrics. If anyone has an idea what other metrics might be
>>>> missing or how we can improve the existing/planned metrics, please
>>>> let us know!
>>>
>>> Hi Karsten,
>>>
>>> Hope I'm not too late.
>>
>> Never too late. :)
>>
>>> This is connected to:
>>>
>>> https://trac.torproject.org/projects/tor/ticket/1826
>>>
>>> It would be useful to know how many cells relays are sending to
>>> streams after they have received a RELAY_END on the stream from the
>>> client.
>>>
>>> On a poor-performing circuit terminating at a fast exit, a RELAY_END
>>> on a stream can result in up to 1000 queued cells getting flushed to
>>> a client that doesn't want them anymore. This wastage of bandwidth
>>> can't be remedied without just tearing down the circuit or resetting
>>> it in some way.
>>>
>>> This metric would tell us how much of this wasted bandwidth there is
>>> on the network and whether it merits action.
>>>
>>> I can implement for your review if you are amenable to the idea.
>>
>> Where would this code be running? Only on the exit node?
>>
> Yes - only on the exit node.

Okay.

>> What would we count? All cells sent to clients vs. cells sent or flushed
>> after seeing a RELAY_END for the contained stream ID?
> 
> Just the latter I think - the absolute number is just as meaningful as 
> comparing it to the total number of cells sent.

Okay.

>> Have you identified the places in the code where we learn about given
>> cells? 
> 
> I *think* it just requires taking the value in the package_window for the 
> stream when the RELAY_END is received and adding that to the counter.

Do you have a patch that prints out a log message whenever we would add
cells or bytes to a counter?

And do you have an exit node to test the new stats code?

--Karsten

>> I can write the code that keeps cell counters, manages
>> measurement periods, and formats results to be written to the logs or to
>> extra-info descriptors.
>>
>> As for getting this stats code merged: We could write the code and ask
>> some friendly exit node operators to run our modified Tor version. If
>> the results turn out to be useful and we decide we want these stats from
>> most or all exit nodes, we can merge the code into master. I think Nick
>> wouldn't want to merge this into 0.2.2.x anymore, so that would be on a
>> 0.2.3.x timeframe. But maybe we're already happy with the results from a
>> few exit nodes that we decide that merging isn't necessary.
>>
>> I like the idea. Unless someone jumps up and says it's a really bad
>> idea, let's do it.
>>
>> Best,
>> --Karsten