[tor-commits] [tech-reports/master] Tweak Kevin's suggestions, comment out some of them.

[karsten at torproject.org]

commit 8142b3e91685e679d4edc4da0a3097ed6ffba125
Author: Karsten Loesing <karsten.loesing at gmx.net>
Date:   Wed Sep 18 12:04:02 2013 +0200

    Tweak Kevin's suggestions, comment out some of them.
---
 2013/torperf2/torperf2.tex |   63 ++++++++++++++++++++++++++++----------------
 1 file changed, 40 insertions(+), 23 deletions(-)

diff --git a/2013/torperf2/torperf2.tex b/2013/torperf2/torperf2.tex
index ceb5677..abc9c39 100644
--- a/2013/torperf2/torperf2.tex
+++ b/2013/torperf2/torperf2.tex
@@ -123,20 +123,29 @@ might be useful?
 \subsubsection{Experiment extensibility considerations}
 
 It should be easy for a user to implement or install an experiment that isn't
-bundled with the core distribution. Installing an experiment should be as
+bundled with the core distribution. Ideally, installing an experiment should be as
 simple as unzipping a folder or config file into an experiments folder.
 Optimally, the Torperf service would detect the new experiment and schedule
 it as soon as practical without additional user intervention.
 
-For implementing, this will require a clearly defined interface for developers
-to work with.
-\footnote{Ideally, this interface would allow experiment implementations in
-any language, not only the language of choice for the core Torperf service.}
-This could be as simple as conforming to a common data format or a tight
-integration with the core service components which could allow lighter weight
-experiment implementations. If the experiment reuses the services
-file server implementation, the service should post-process the result
-file(s) to add any additional serverside timing information available.
+% Commented out Kevin's text by Karsten on 2013-09-18:
+% For implementing, this will require a clearly defined interface for developers
+% to work with.
+% \footnote{Ideally, this interface would allow experiment implementations in
+% any language, not only the language of choice for the core Torperf service.}
+% This could be as simple as conforming to a common data format or a tight
+% integration with the core service components which could allow lighter weight
+% experiment implementations. If the experiment reuses the service's
+% file server implementation, the service should post-process the result
+% file(s) to add any additional server-side timing information available.
+%
+% Reason: We're not trying to build a generic experiment runner, but a
+% specific Tor performance measurement tool.  Most experiments will depend
+% on various Torperf components to start a tor process and parse tor
+% events to storing results in the database.  There's no (reasonable) way
+% for those experiments to interface with Torperf components if they don't
+% speak Torperf's programming language.  If people want to add a simple
+% experiment, they can write a wrapper for using their tool in Torperf.
 
 \subsubsection{User-defined tor version or binary}
 
@@ -148,20 +157,20 @@ Similarly, it should be easy to point to a custom tor binary to use for
 measurements.
 
 It should be possible to run different experiments with different tor
-versions or binaries in the same Torperf service instance.
+versions or binaries in the same Torperf service instance.%
+\footnote{Loosely specified tor versions, e.g. 'latest-stable, git:master,
+$\geq$ 0.2.4' (following a well-defined schema),
+would be a boon in reducing error prone configuration updates everytime
+there are new tor versions released.}
 An experiment should be able to be configured to run for multiple
 versions of tor, potentially simultaneously, to spot performance change
 across tor releases. Note that the tor version should be contained in the
 results.
-\footnote{Loosely specified tor versions, e.g. 'latest-stable, git:master,
-\textasciitilde>0.2.4',
-would be a boon in reducing error prone configuration updates everytime
-there are new tor versions released.}
 
 It might be beneficial to provide a mechanism to download and verify the
 signature of new tor versions as they are released. The user could speficy
 if they plan to test stable, beta or alpha versions of tor with their
-torperf instance.
+Torperf instance.
 
 \subsubsection{User-defined third-party software version or binary}
 
@@ -206,15 +215,18 @@ plausible, too.
 
 \subsubsection{Results web interface}
 
-Torperf should provide all its results via a public web interface.
+Torperf should provide all its results via a public web interface.%
 \footnote{It is a requirement to also provide a machine readable representation
 to enable easier development of custom visualisations of the result data.}
 This includes measured timestamps as well as any meta data collected in
 an experiment.
 Results should be available for all measurements as well as for a subset
 specified by measurement time or filtered by other criteria. The interface
-should highlight experiment failures and provide a convenient mechanism to
-run a set of experiments on demand.
+should highlight experiment failures.
+% Commented out Kevin's text by Karsten on 2013-09-18:
+% and provide a convenient mechanism to
+% run a set of experiments on demand.
+% Reason: The web interface has to be read-only for security reasons.
 
 \subsubsection{Results accumulator}
 
@@ -411,14 +423,19 @@ client processes, using a previously configured tor version or binary.
 \item[tor controller] Connect to tor's control port, force-set guards if
 required by the experiment, register for and handle asynchronous events,
 set up and tear down hidden services.
-\item[experiment scheduler] Run experiments following a schedule. Must also
-handle immediate scheduling requests by users.
+\item[experiment scheduler] Run experiments following a schedule.
+% Commented out Kevin's text by Karsten on 2013-09-18:
+% Must also
+% handle immediate scheduling requests by users.
+% Reason: Following a pre-defined schedule should be sufficient.  See also
+% above why users shouldn't trigger new measurements from the web
+% interface.
 \item[experiment runner] Handle the execution of a scheduled experiment. It
 should take care of the entire experiments lifecycle and reserve an
 exclusive tor instance for its lifetime. Finally, collect the results
 and post-process them (if applicable) before saving the data to the
-results database.If the experiment fails it should save as much information
-as possible for future inspection.
+results database. If the experiment fails it should be possible to track
+down the reason for failure from looking at the experiment results.
 \item[request scheduler] Start new requests following a previously
 configured schedule.
 \item[request runner] Handle a single request from creation over various