[tor-bugs] #24737 [Core Tor/Tor]: Recommend a MaxMemInQueues value in the Tor man page

[Tor Bug Tracker & Wiki]

#24737: Recommend a MaxMemInQueues value in the Tor man page
----------------------------+------------------------------------
 Reporter:  starlight       |          Owner:  (none)
     Type:  defect          |         Status:  new
 Priority:  Medium          |      Milestone:  Tor: 0.3.2.x-final
Component:  Core Tor/Tor    |        Version:
 Severity:  Normal          |     Resolution:
 Keywords:  doc, tor-relay  |  Actual Points:
Parent ID:                  |         Points:  0.5
 Reviewer:                  |        Sponsor:
----------------------------+------------------------------------

Comment (by starlight):

 I think suggestions in comment 6 and 7 are a bit conservative (but ok) and
 still like my approx 40% of available memory per instance.  So on my 4G
 machine thinking ~1G for the kernel, I set MaxMemInQueues=1024MB for one
 relay instance and have some room for a some other daemons.  With this
 setting tor daemon 0.2.9.14 goes to 1.5GB under heavy load (old slow CPU
 and medium-fast FiOS connection, YMMV) and when hit with a known sniper
 attack it went to 2GB and survived with Tor's OOM logic killing a 1GB
 circuit (event log entries above).  Leaves quite a bit of space for socket
 buffer memory and about 500-700MB of other daemons.  Note I prefer
 DisableAllSwap=1 and recommend it strongly, so all Tor daemon memory will
 fall in the Unevictable/Mlocked accounting and _cannot_ be paged to disk
 (a detrimental behavior no doubt).

 Put another way, MMIQ=1G -> daemon 2GB (80%) plus socket-buffer-delta
 guess 500MB or 2.5GB total budget (100%) for the instance.

 I see kernel SLAB around 900MB (buffer frees tuned lazily with ~7000
 active TCP connections at the time of observation, peak around ~9000
 connections).

 On a 4G machine running just Tor and nothing else, I'd take 40% of 3G and
 get MMIQ=1228MB.

 Don't forget sysctl.conf

 vm.min_free_kbytes = 262144

 which causes linux to attempt to keep 1/4 GB of memory free.  Linux will
 take aggressive action to page-out idle memory and free cached files when
 this threshold is hit--it's not an absolute impediment to allocations.
 The idea is a huge sudden burst of network traffic will rapidly chew up
 free memory for socket buffers, and if /proc/meminfo:MemFree hits zero and
 the kernel needs to allocate memory while servicing a network interrupt,
 the systems will OOPS/crash.  So one wants linux to maintain a nice
 cushion against hard memory exhaustion.  /proc/meminfo:Cached not-dirty
 memory the easiest target for obtaining true free memory, but Cached pages
 cannot be converted to MemFree during interrupt service--takes some time,
 i.e. a few hundred microseconds to a couple of milliseconds depending on
 how busy the scheduler is.

 On an 8GB machine I'd still take 1G for the kernel and then 40% of 7G for
 MaxMemInQueues=2800MB.  Two daemons MMIQ=1400MB.  On big memory systems
 (16GB and up) I don't bother setting MMIQ higher than 4096MB or 4G for an
 instance.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/24737#comment:8>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online