[metrics-bugs] #21087 [Metrics/CollecTor]: What is @source?

[Tor Bug Tracker & Wiki]

#21087: What is @source?
-------------------------------+------------------------------
 Reporter:  atagar             |          Owner:  metrics-team
     Type:  defect             |         Status:  new
 Priority:  Medium             |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  metrics-help       |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+------------------------------
Changes (by karsten):

 * keywords:   => metrics-help
 * type:  enhancement => defect


Comment:

 Hi Damian!

 This looks like a truncated descriptor (with the last characters being
 `"reject *:"`) and another complete descriptor obtained from cached
 descriptor files (starting with `"@uploaded-at"`).

 CollecTor could indeed be smarter about separating those two descriptors.
 What it does is look for a descriptor start `"^router "` and the next
 descriptor end `"\nrouter-signature\n"` and consider anything between the
 descriptor.  Here's the
 [https://gitweb.torproject.org/collector.git/tree/src/main/java/org/torproject/collector/relaydescs/RelayDescriptorParser.java#n233
 relevant code].

 We did not notice this issue before, because we'd have discarded the
 descriptor after finding that it doesn't pass metrics-lib's parser.  But
 [https://lists.torproject.org/pipermail/tor-dev/2016-August/011271.html as
 of five months ago], we're keeping those descriptors anyway.

 A possible fix would be to check whether there's another `startToken` (or
 rather another string `"\n" + startToken`) before `sigToken`, and if there
 is, treat that substring as separate descriptor.  In fact, there could be
 several truncated descriptors before the first complete descriptor.

 This could be something that a new volunteer could hack on.

 Thanks for the report!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21087#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online