[metrics-bugs] #33502 [Metrics/CollecTor]: Do not let appended descriptor files grow too large

[Tor Bug Tracker & Wiki]

#33502: Do not let appended descriptor files grow too large
-------------------------------+------------------------------
 Reporter:  karsten            |          Owner:  karsten
     Type:  enhancement        |         Status:  needs_review
 Priority:  Medium             |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:                     |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+------------------------------

Comment (by karsten):

 Here's another option: rather than append multiple descriptors to a single
 flat file we could produce a tarball containing the few hundred or
 thousand descriptor files. Basically,

   `https://collector.torproject.org/recent/relay-descriptors/server-
 descriptors/2020-03-10-14-05-00-server-descriptors`

 containing 596 descriptors concatenated to a 1.4 MiB file would then be
 replaced by

   `https://collector.torproject.org/recent/relay-descriptors/server-
 descriptors/2020-03-10-14-05-00-server-descriptors.tar`

 containing 596 descriptor files.

 Advantage over the approach sketched out above would be that we wouldn't
 have three output file formats anymore (flat file with 1 descriptor, flat
 file with >= 1 descriptors, tarball). Disadvantage might be that
 processing tarballs can be less convenient than processing flat files.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/33502#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online