[metrics-bugs] #21219 [Metrics/CollecTor]: collector should rm data from out/ after archiving

[Tor Bug Tracker & Wiki]

#21219: collector should rm data from out/ after archiving
-------------------------------+--------------------------
 Reporter:  tom                |          Owner:  iwakeh
     Type:  enhancement        |         Status:  accepted
 Priority:  Medium             |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:                     |  Actual Points:
Parent ID:  #20546             |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+--------------------------
Changes (by iwakeh):

 * status:  new => accepted
 * owner:  metrics-team => iwakeh
 * parent:   => #20546


Comment:

 Replying to [comment:1 karsten]:
 > You're right.  It's safe to delete descriptors that are at least N = 2
 months old.  In fact, on the main CollecTor instance I usually delete
 descriptors that are at least N = 1.5 months old.  But I do this manually,
 because of https://xkcd.com/1205/ and https://xkcd.com/1319/, and more
 importantly because I'm afraid of messing up with scripts and accidentally
 deleting all descriptors.

 I think you accidentally pasted the wrong link :-)
 The description is
 [https://gitweb.torproject.org/collector.git/tree/INSTALL.md#n255 here].

 >
 > I admit that these reasons don't fully apply anymore with three
 CollecTor instances running.  Would you submit a patch to delete
 descriptors older than N = 2 months, possibly even with a configurable N?
 >
 > https://gitweb.torproject.org/collector.git/tree/src/main/resources
 /create-tarballs.sh
 >
 > And if you have ideas for making that script even less copy-and-pasty,
 please feel free to tweak it!

 One of the planned improvements is to integrate all scripted maintenance
 into java and get rid of scripting.
 We're planning and working on these steps already, see #20518, #20546. So,
 please check the back first to avoid duplicate work.  Setting parent to
 #20546.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21219#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online