[metrics-bugs] #22983 [Metrics/metrics-lib]: add a descriptor interface and implementation for web-logs

[Tor Bug Tracker & Wiki]

#22983: add a descriptor interface and implementation for web-logs
---------------------------------+-----------------------------------
 Reporter:  iwakeh               |          Owner:  metrics-team
     Type:  enhancement          |         Status:  needs_revision
 Priority:  Medium               |      Milestone:  metrics-lib 2.1.0
Component:  Metrics/metrics-lib  |        Version:
 Severity:  Normal               |     Resolution:
 Keywords:                       |  Actual Points:
Parent ID:                       |         Points:
 Reviewer:                       |        Sponsor:
---------------------------------+-----------------------------------

Comment (by iwakeh):

 Replying to [comment:14 karsten]:
 > I'm game. Your suggestion doesn't at all sound crazy. The only reason I
 pushed back was that such a change requires a lot of consideration and
 discussion and shouldn't be included together with a new feature but on
 its own. A few thoughts:

 Fine :-)

 >  - How about we include a second package with a second set of interfaces
 for CollecTor and other hypothetical applications providing and/or
 sanitizing descriptors? Ideally, applications that simply consume
 descriptors wouldn't have to worry about that second set. And
 implementation classes could implement interfaces from both sets.

 Makes sense; I would suggest `org.torproject.descriptor.internal`.

 >  - I agree that the discussion of JSON handling parts in CollecTor and
 metrics-lib was related. Another aspect there is metrics-web that will
 soon need to parse CollecTor's index.json, too, in order to implement
 #22836.

 Yep.

 >  - I'm not opposed to including the sanitizing code in metrics-lib if
 it's hidden from the descriptor-consuming interfaces. CollecTor will still
 contain all the logic for timing downloads. But as soon as we put the
 sanitized bridge descriptor specification into metrics-web, CollecTor
 won't be the only place anymore that needs changing when we change
 sanitizing code anyway. So, it doesn't really matter whether the
 sanitizing code is in CollecTor or metrics-lib. I could imagine that we
 can remove some duplicate code by putting everything directly related to
 messing with descriptors in a single place.

 Agreed.  This would also allow tests that could catch problems when
 sanitation changes etc.

 >  - We should focus on one change at a time. That is, we could either
 start with adding web logs without any sanitizing support and duplicate
 some code in CollecTor. Or we could start with this discussion of
 extending scope of metrics-lib and put the web logs extension on hold. I'm
 fine with either approach.

 Well, as for the next steps I would go ahead and add a section to metrics-
 lib README.md about the `internal` API package, which will condense and
 finalize the discussion (new ticket?).
 Then, I'd also like to start immediately putting the definitions/design
 there into code for webstats.  The code for the webstats module is already
 there and just needs to be 'distributed' across products.
 The public-API addition to metrics-lib is defined (i.e., the above with
 the suggestions from your comment and w/o any sanitation related).  The
 additional internal-API is internal and only used in CollecTor's new
 webstats module, which leaves a lot freedom to adapt things, if necessary.

 > There, I didn't fully think this through, but I didn't want to delay
 this any further in case you want to give this more thoughts. I'll keep
 thinking about this a little, as time permits, but please let me know how
 you want to continue, and I'll try to help move it forward in that
 direction. Thanks!

 Thanks, too!
 I think, this is a good step forward.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/22983#comment:15>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online