[tor-bugs] #31204 [Metrics/CollecTor]: Extend file objects in index.json to include descriptor types, publication times, and file digests

[Tor Bug Tracker & Wiki]

#31204: Extend file objects in index.json to include descriptor types, publication
times, and file digests
-------------------------------+--------------------------
 Reporter:  karsten            |          Owner:  karsten
     Type:  enhancement        |         Status:  accepted
 Priority:  Medium             |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:                     |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+--------------------------

Comment (by atagar):

 > I started working on this today.

 Wonderful, thanks Karsten! I've been making progress toward a stem
 collector module on my end too. The sha256 attribute will be particularly
 helpful to determine if download requests can be short circuited or not.

 https://gitweb.torproject.org/user/atagar/stem.git/tree/stem/descriptor/collector.py?h=collector#n262

 > Reading tarballs to find out descriptor types and publication times is
 really time consuming.

 What creates the tarballs? Just naive spit balling on my end, but @type
 annotations get added by metrics code somewhere so I wonder if the process
 that determines tarball @types and filenames can be shared and used by the
 indexer.

 Feel free to disregard if this is silly. I'm pretty fuzzy on how
 CollecTor's architected. :P

 > I wonder how it will work out in practice that these new fields will be
 blank for 10-20 minutes for newly created tarballs

 Ooph. That would definitely be confusing for users.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/31204#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online