[tor-bugs] #34031 [Metrics/Onionperf]: Figure out warning about unknown error type when exporting .tpf file

[Tor Bug Tracker & Wiki]

#34031: Figure out warning about unknown error type when exporting .tpf file
---------------------------------------+--------------------------------
 Reporter:  karsten                    |          Owner:  acute
     Type:  defect                     |         Status:  assigned
 Priority:  Medium                     |      Milestone:
Component:  Metrics/Onionperf          |        Version:
 Severity:  Normal                     |     Resolution:
 Keywords:  metrics-team-roadmap-2020  |  Actual Points:
Parent ID:                             |         Points:  2
 Reviewer:                             |        Sponsor:  Sponsor59-must
---------------------------------------+--------------------------------

Comment (by acute):

 To start off, a bit of background:
 The `ERRORCODE` field present in the tpf files combines error codes from
 both Tgen and Tor's json output. To produce `ERRORCODE`, we combine three
 fields: Tgen's `error_code` field, Tor's `failure_reason_local` (if
 present), and `failure_reason_remote` (if present, and only if
 failure_reason_local is present, too). While all the error codes are
 present individually in the json output, the `ERRORCODE` field is produced
 by matching between the two on the stream's source port, and is output
 only in the tpf file.

 This is something to keep in mind when we remove tpf files in the future.

 So, to confirm this bug only affects tpf files, and not json, and we are
 missing the following cases:


 Tgen Error: `PROXY` + Tor Error Local: `END` + Tor Error Remote:
 `CONNRESET` = `PROXY_END_CONNRESET`(missing key PROXY_END_CONNRESET)
 Tgen Error: `PROXY` + Tor Error Local: `END` + Tor Error Remote:
 `INTERNAL` = `PROXY_END_INTERNAL`(missing key PROXY_END_INTERNAL)
 Tgen Error: `PROXY` + Tor Error Local: `END` + Tor Error Remote: `MISC` =
 `PROXY_END_MISC` (missing key PROXY_END_MISC)

 We could add the following to our existing dictionary (in line with how we
 previously named these, see #29787), and this would fix the tpf export:
 {{{
 PROXY_END_CONNRESET: TOR/END/CONNRESET
 PROXY_END_INTERNAL: TOR/END/INTERNAL
 PROXY_END_MISC: TOR/END/MISC
 }}}
 I can attach a patch for the above.

 We additionally see this case:

 Tgen Error: None + Tor Error Local: `TIMEOUT` + Tor Error Remote: None =
 `_TIMEOUT` (missing key ERRORCODE)
 This happens when Tgen never sees an an error, but the stream reports a
 timeout, and results in warnings like the following:
 {{{
 2020-05-06 20:05:00 1588795500.444442 [onionperf] [WARNING] KeyError while
 exporting torperf file, missing key ''ERRORCODE'', skipping transfer
 'transfer1m:2189'
 }}}

 Not sure how to handle cases where the Tgen transfer appears to complete
 with no error, but Tor repors a timeout - should we record this as an
 error, even if the transfer completed?

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/34031#comment:11>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online