[tor-dev] Torperf implementation considerations (was: Torperf)

[Karsten Loesing]

On 9/18/13 3:49 AM, Kevin Butler wrote:
>> Executing scripts and reading stdout/stderr is probably too low-level.
>> I think we need a Python/Twisted (or whatever language Torperf will be
>> written in) interface for running an experiment and retrieving results.
>
> You're probably right on stdout/err being too low level, but most
> experiments would be reusing the provided implementation, just wrapping
> them in a simple manner. Anyway I think there's a language agnostic way to
> get this working, without forcing any extensibility on matching Torperfs
> language of choice. I've tried to be pretty generic in my attached changes.
> :)

I don't see how we could make new experiments language agnostic.  These
new experiments will want to re-use Torperf classes/components, which
they can't do if they're just "shell scripts".  They need to implement
an interface provided by Torperf, and Torperf needs to provide
functionality via an interface that the experiments understand.  If an
experiment is in fact just a shell script, it should be trivial to write
a wrapper class to call that.  But I think that's the exception, not the
rule.

Or maybe we have a different understanding of an "experiment".  Can you
give an example for an experiment that is not listed in Section 3 of the
design document and state how you'd want to integrate that into Torperf
without touching its sources?

>> Well, thanks for your input!  As I said above, it would help a lot if
>> you added these ideas to the appropriate sections of the design document.
>
> Please see attached.

Awesome!  I applied your patch, though I tweaked some parts and
commented out other parts, explaining my reasons in the LaTeX sources.
Happy to discuss these things further if you want!

I also added an Appendix A with suggested data formats.  Maybe these
data formats make it clearer what I'd expect as output from an experiment.

https://people.torproject.org/~karsten/volatile/torperf2.pdf

If you have further suggestions how to improve the requirements or
design, please send me a new patch, and I'll apply it and comment on it.
 Thanks!

All the best,
Karsten