[metrics-bugs] #22990 [Metrics/metrics-lib]: Add a heartbeat log message indicating progress and estimated time left

[Tor Bug Tracker & Wiki]

#22990: Add a heartbeat log message indicating progress and estimated time left
-------------------------------------+--------------------------
     Reporter:  karsten              |      Owner:  metrics-team
         Type:  enhancement          |     Status:  new
     Priority:  Medium               |  Milestone:
    Component:  Metrics/metrics-lib  |    Version:
     Severity:  Normal               |   Keywords:
Actual Points:                       |  Parent ID:
       Points:                       |   Reviewer:
      Sponsor:                       |
-------------------------------------+--------------------------
 It would be really useful to know how long it still takes to read a bunch
 of descriptors, as this can range from minutes to hours or even days. In
 some cases this information would help to decide whether to continue or
 abort and try something different.

 How about we start reading descriptors by walking files in the given
 directories, summing up their bytes, and using that total sum to
 periodically write a log message with progress and estimated time left?

 Once per minute on debug level would be sufficient. And it's okay if it's
 not super precise and gets confused by compression or other factors. Just
 a rough number that gets more precise over time.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/22990>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online