[metrics-bugs] #22026 [Metrics]: Add new raw document types

[Tor Bug Tracker & Wiki]

#22026: Add new raw document types
-------------------------+------------------------------
 Reporter:  irl          |          Owner:  metrics-team
     Type:  enhancement  |         Status:  new
 Priority:  Medium       |      Milestone:
Component:  Metrics      |        Version:
 Severity:  Normal       |     Resolution:
 Keywords:               |  Actual Points:
Parent ID:               |         Points:
 Reviewer:               |        Sponsor:
-------------------------+------------------------------

Comment (by iwakeh):

 Replying to [comment:5 karsten]:
 > I'm opposed to making this a CollecTor ticket/module for the following
 reasons:
 >
 >  - The main purpose of CollecTor is to reliably collect data from the
 Tor network, not to provide convenient access to that data.

 That's a fine definition/specification for CollecTor's purpose.  Should be
 added somewhere prominently.

 > ... If both things can be achieved at the same time, great, but I see
 the risk of adding quite some complexity for such a descriptor query
 module that would at least shift focus away from descriptor completeness.
 >  - A good descriptor retrieval tool would also retrieve descriptors from
 weeks, months, or even years ago.  But CollecTor does not have a database
 with all descriptors in its archive, nor does it need one for its main
 purpose stated above.
 >  - CollecTor's current web interface is pretty dumb, which I consider a
 feature.  It's good that we don't need a Tomcat for CollecTor and can
 serve all contents using Apache.

 Aggreed, I actually suggested CollecTor, b/c of a lack of matching
 components in the Metrics product list.
 In total, it really is a new retrieval tool and way more than a component
 that can be attached to an existing Metrics product.

 >
 > I'd prefer a solution where we either extend Onionoo to provide raw
 descriptor contents (contained in JSON documents) or build a new
 descriptor retrieval tool that takes descriptors from CollecTor, imports
 them into a database, and provides a query interface for that.  I don't
 have a clear preference for either of the two solutions.

 I vote for the new retrieval tool.

 >
 > I'm moving this ticket to the Metrics supercomponent, admitting that
 it's somewhat out of scope for existing subcomponents and hence might not
 happen in the near future.

 We could create a trac component for tickets/ideas like this one?  Maybe,
 'Metrics Ideas' (there are surely better naming options).

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/22026#comment:6>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online