[tor-dev] Trouble with onionperf visualize and S61 performance experiments

[George Kadianakis]

Hello Karsten,

hope you are doing well!

I've been working on the S61 performance experiments [0] and I would appreciate
some help with onionperf.

I have done various onionperf measurements using something the following command:
  $ onionperf measure -i --tgen ~/tgen/build/src/tgen --tor ~/onionperf/tor/src/app/tor --drop-guards 10

I put each of the measurements on a different directory and now I want
to analyze them and derive the CDF-TTFB graphs etc. I attempted doing
that using the following calls:

 $ onionperf analyze --tgen ./tgen-client/onionperf.tgen.log --torctl ./tor-client/onionperf.torctl.log
 $ onionperf visualize --data onionperf.analysis.json.xz "test"

Unfortunately, the 'visualize' call can fail for the attached 'onionperf-mbps.json.xz':

  $ onionperf visualize --data onionperf.analysis.json.xz "Test Measurements"
  2020-11-03 15:51:31 1604411491.540736 [onionperf] [INFO] loading analysis results from /user/tmp/onionperf/analysis/onionperf.analysis.json.xz
  2020-11-03 15:51:31 1604411491.577864 [onionperf] [INFO] done!
  2020-11-03 15:51:31 1604411491.586845 [onionperf] [INFO] NumExpr defaulting to 8 threads.
  /user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py:251: UserWarning: Attempting to set identical left == right == -1e-06 results in singular transformations; automatically expanding.
  /user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py:251: UserWarning: Attempting to set identical left == right == -1e-06 results in singular transformations; automatically expanding.
  Traceback (most recent call last):
    File "/user/.local/bin/onionperf", line 4, in <module>
      __import__('pkg_resources').run_script('OnionPerf==0.8', 'onionperf')
    File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 650, in run_script
      self.require(requires)[0].run_script(script_name, ns)
    File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1453, in run_script
      exec(script_code, namespace, namespace)
    File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 622, in <module>
    File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 382, in main
    File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 522, in visualize
    File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 48, in plot_all
    File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 205, in __plot_throughput_ecdf
    File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 235, in __draw_ecdf
    File "/usr/lib/python3/dist-packages/pandas/core/frame.py", line 5000, in dropna
      raise KeyError(list(np.compress(check, subset)))
  KeyError: ['mbps']

Then for another onionperf run (attached as 'onionperf-44028.json.xz') it gave me a different error:

  $ onionperf visualize --data onionperf.analysis.json.xz "Test Measurements"
  2020-11-03 15:50:03 1604411403.946028 [onionperf] [INFO] loading analysis results from /user/tmp/onionperf/analysis/onionperf.analysis.json.xz
  2020-11-03 15:50:03 1604411403.976088 [onionperf] [INFO] done!
  Traceback (most recent call last):
    File "/user/.local/bin/onionperf", line 4, in <module>
      __import__('pkg_resources').run_script('OnionPerf==0.8', 'onionperf')
    File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 650, in run_script
      self.require(requires)[0].run_script(script_name, ns)
    File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1453, in run_script
      exec(script_code, namespace, namespace)
    File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 622, in <module>
    File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 382, in main
    File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 522, in visualize
    File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 38, in plot_all
    File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 129, in __extract_data_frame
  KeyError: '44028'

Because of the two errors above I'm unable to proceed with producing any
results.  Any idea what I might be doing wrong? Is there any other data
you might need to help me with this?

Thanks a lot!

[0]: https://trac.torproject.org/projects/tor/wiki/org/roadmaps/CoreTor/PerformanceExperiments
     https://gitlab.torproject.org/tpo/core/tor/-/issues/40157

-------------- next part --------------
A non-text attachment was scrubbed...
Name: onionperf-mbps.json.xz
Type: application/x-xz
Size: 137320 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20201103/16295409/attachment-0002.xz>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: onionperf-44028.json.xz
Type: application/x-xz
Size: 109660 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20201103/16295409/attachment-0003.xz>