[tor-bugs] #29374 [Metrics/Onionperf]: Analysis files sometimes present negative numbers in the payload_progress field

[Tor Bug Tracker & Wiki]

#29374: Analysis files sometimes present negative numbers in the payload_progress
field
-----------------------------------+------------------------------
 Reporter:  irl                    |          Owner:  metrics-team
     Type:  defect                 |         Status:  new
 Priority:  Medium                 |      Milestone:
Component:  Metrics/Onionperf      |        Version:
 Severity:  Normal                 |     Resolution:
 Keywords:  acute-2019-q1-planned  |  Actual Points:
Parent ID:                         |         Points:
 Reviewer:                         |        Sponsor:
-----------------------------------+------------------------------

Comment (by acute):

 Numbers in the payload progress field are calculated based on a
 unix_ts_start property of a transfer, which is in turned calculated by
 subtracting the time elapsed recorded by tgen from unix_ts_end (which
 holds the timestamp of the latest line in the transfer log). In case of
 completed transfers this works ok because tgen reports elapsed seconds to
 completion. In case of errors tgen does not report this, and sp the
 unix_ts_start property is incorrectly calculated (hence the negative
 numbers).

 I propose a patch to record unix_ts_start timestamp directly from the log
 line corresponding to the tgen socket creation for that transfer. The line
 looks like this:

 ```2019-04-01 03:15:10 1554084910.250433 [info] [shd-tgen-transport.c:234]
 [_tgentransport_newHelper] Initiated transport to socks proxy at
 localhost:127.0.0.1:30231 ```

 For completed transfers, this would be overwritten with the calculated
 value based on the tgen report, which is more acurate (usec). For
 transfers which do not complete, this would be used to calculate the
 payload progress field instead.

 I have a patch ready on my development machine, but need more tgen logs to
 test it with.

 Any suggestions for improvement to the above mechanism would be
 appreciated.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29374#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online