[metrics-bugs] #32660 [Metrics/Onionoo]: onionoo-backend is killing the ganeti cluster

[Tor Bug Tracker & Wiki]

#32660: onionoo-backend is killing the ganeti cluster
-----------------------------+------------------------------
 Reporter:  anarcat          |          Owner:  metrics-team
     Type:  defect           |         Status:  new
 Priority:  Medium           |      Milestone:
Component:  Metrics/Onionoo  |        Version:
 Severity:  Normal           |     Resolution:
 Keywords:                   |  Actual Points:
Parent ID:                   |         Points:
 Reviewer:                   |        Sponsor:
-----------------------------+------------------------------

Comment (by karsten):

 Here's an attempt to reduce writes:

 {{{
 2019-12-05 09:45:45,384 INFO o.t.m.o.cron.Main:216 Document store
     7 list operations performed
     951,149 files listed
     56,448 files stored
     1.4 GiB stored
     912,471 files not rewritten   <--
     641.3 MiB not rewritten       <--
     983,247 files retrieved
     2.8 GiB retrieved
     0 files removed
 }}}

 The idea is to read existing files before writing them, compute their
 digests, compute digests of the content to be written, and then abort the
 write if nothing has changed.

 It looks like this saves us from overwriting a lot of unchanged files. It
 doesn't save as many bytes, but that might be secondary. Of course, this
 patch comes at the price of doing more reads and digest computations. It
 might be worth it, just saying that it's not free.

 I'll bring this patch up at today's team meeting. We should discuss this
 and possibly merge it before doing new deployments on `onionoo-
 backend-0[12]`.

 And then we could look at those graphs once more to see if we need to
 tweak things more. Thanks for making these graphs!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/32660#comment:8>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online