[metrics-bugs] #29374 [Metrics/Onionperf]: Analysis files sometimes present negative numbers in the payload_progress field

[Tor Bug Tracker & Wiki]

#29374: Analysis files sometimes present negative numbers in the payload_progress
field
-----------------------------------+------------------------------
 Reporter:  irl                    |          Owner:  metrics-team
     Type:  defect                 |         Status:  new
 Priority:  Medium                 |      Milestone:
Component:  Metrics/Onionperf      |        Version:
 Severity:  Normal                 |     Resolution:
 Keywords:  acute-2019-q1-planned  |  Actual Points:
Parent ID:                         |         Points:
 Reviewer:                         |        Sponsor:
-----------------------------------+------------------------------

Comment (by karsten):

 Some thoughts on this, since I have been looking at tgen logs for a while
 now:

  - The tgen log line you mention does not contain anything to identify the
 transfer. Would it be easier to use the next tgen log line which at least
 contains the source port? `2019-02-11 00:13:00 1549843980.555482 [info]
 [shd-tgen-transport.c:203] [_tgentransport_changeState] transport
 TCP,16,localhost:127.0.0.1:43810,localhost:127.0.0.1:41204,op-
 ab.onionperf.torproject.net:137.50.19.2:80,state=CONNECT,error=NONE moving
 from state CONNECT to state INIT`; and would it still be accurate enough?
  - What's the reason for overwriting this timestamp with the calculated
 value for ''complete'' transfers? This log line has usec granularity, too
 (1554084910'''.250433'''). I think it would be easier to compare partial
 progress of complete and incomplete transfers if we'd use the same
 timestamps for calculating them.
  - Do we know of any tools ''relying'' on `payload_progress` numbers being
 negative for incomplete downloads? I admit that this would be a weird use
 of the data. Still, maybe worth checking if we're using them anywhere
 ourselves and probably worth writing something in the next release notes.
  - You'll find more tgen logs for testing at https://op-
 ab.onionperf.torproject.net/ .

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29374#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online