[tor-bugs] #24737 [Core Tor/Tor]: oft given MaxMemInQueues advice is wrong

[Tor Bug Tracker & Wiki]

#24737: oft given MaxMemInQueues advice is wrong
------------------------------+--------------------
     Reporter:  starlight     |      Owner:  (none)
         Type:  defect        |     Status:  new
     Priority:  Medium        |  Milestone:
    Component:  Core Tor/Tor  |    Version:
     Severity:  Normal        |   Keywords:
Actual Points:                |  Parent ID:
       Points:                |   Reviewer:
      Sponsor:                |
------------------------------+--------------------
 due to recent DOS attacks much incorrect advice has been tossed around on
 tor-relays regarding the application of `MaxMemInQueues`

 many seem to believe that MaxMemInQueues should be set to 75-80% of
 available memory but this is painfully (in the sense of OOM crashes)
 incorrect

 proper advice is to set MaxMemInQueues to 45% of physical memory available
 for the instance, assuming DisableAllSwap=1 is also in effect; 40% is a
 safer, more conservative value

 one of my relays configured with MaxMemInQueues=1024MB recently emitted

 {{{
 We're low on memory.  Killing circuits with over-long queues. (This
 behavior is controlled by MaxMemInQueues.)
 Removed 1063029792 bytes by killing 1 circuits; 21806 circuits remain
 alive. Also killed 0 non-linked directory connections.
 }}}

 after which the tor daemon was observed to consume precisely 2GB per /proc
 /<tor-pid>/status:VmRSS

 the aforementioned incorrect advice was followed in #22255 and the
 operator continues to experience OOM failures

 another mitigation is to establish conservative linux memory management
 with the sysctl settings

 vm.overcommit_memory = 2
 vm.overcommit_ratio = X

 where X is set such that /proc/memifo:CommitLimit is approximately 80% of
 physical memory (90% if 16GB or more is present)

 The settings will prevent sparse-memory applications from running (e.g.
 ASAN instrumented code), but is appropriate for dedicated tor relays
 systems.  Effectively disables OOM killer and should result in graceful
 memory exhaustion behavior, though I have not investigated tor daemon
 response in the face of malloc() fails returning null pointers.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/24737>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online