[tor-bugs] #25372 [Core Tor/Tor]: relay: Allocation for compression goes very high

[Tor Bug Tracker & Wiki]

#25372: relay: Allocation for compression goes very high
-----------------------------+---------------------------------------------
     Reporter:  dgoulet      |      Owner:  (none)
         Type:  defect       |     Status:  new
     Priority:  High         |  Milestone:  Tor: 0.3.4.x-final
    Component:  Core         |    Version:
  Tor/Tor                    |
     Severity:  Normal       |   Keywords:  tor-relay, compression, tor-dos
Actual Points:               |  Parent ID:
       Points:               |   Reviewer:
      Sponsor:               |
-----------------------------+---------------------------------------------
 My relay just OOMed some circuits with filled up queue (#25226) but then a
 useful log was printed showing that the compress total allocation is huge.

 {{{
 Feb 27 20:02:55.718 [notice] We're low on memory (cell queues total alloc:
 232279872 buffer total alloc: 1937408, tor compress total alloc: 878586075
 rendezvous cache total alloc: 4684497). Killing circuits withover-long
 queues. (This behavior is controlled by MaxMemInQueues.)
 }}}

 That `878586075 = ~838MB`. My relay is hovering around 1.4GB of RAM right
 now which means ~60% of the RAM used is in the compression subsystem.

 I'm not sure where it all comes, the relay is serving directory data but I
 have my doubt that *compressed*, it comes down to 800+ MB...

 Datapoint:

 {{{
 $ du -sh diff-cache/
 131M    diff-cache/
 }}}

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25372>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online