huge pages, was where are the exit nodes gone?

Tue Apr 13 10:58:32 UTC 2010

     On Tue, 13 Apr 2010 05:16:33 -0500 (CDT) I wrote:
>     On Tue, 13 Apr 2010 11:04:36 +0200 Olaf Selke <olaf.selke at blutmagie.de>
>wrote:
>>Scott Bennett wrote:
>>
>>>      Either I forgot (probable) or you didn't mention before (less probable)
>>> that you had moved it to a newer machine.  Whatever you're running it on,
>>> superpages or LINUX's "huge" pages ought to speed tor up considerably by
>>> drastically reducing TLB misses.  (I wasn't suggesting that you revert to
>>> older hardware.  I was thinking that you were still running tor on the Xeon-
>>> based machine.)
>>
>>I just setup hugepages (1024 pages a 2 MB) according this hint
>>http://www.pythian.com/news/1326/performance-tuning-hugepages-in-linux/
>
>     Very interesting article.  Thanks for the URL.  Of course, not being
>a LINUX user, I have no idea what the acronyms for various LINUX kernel
>features mean, and I have mercifully been free of any involvement with
>Oracle for ~17 years, so ditto for the Oracle stuff. :-)
>     One matter of concern, though, is the mention of a page size of 2 MB.
>Intel x86-style CPUs offer a 2 MB page size *only* for instruction (a.k.a.
>text) segments, not for data or stack segments, so I'm not sure what LINUX
>is doing with that.  (See also the last line of the following bit of
>output.)
>>
>>anonymizer2:~# echo 1024 > /proc/sys/vm/nr_hugepages
                      ^^^^
>>anonymizer2:~# cat /proc/meminfo | grep -i hugepage
>>HugePages_Total:     126
                       ^^^
     Apparently, telling it reserve 1024 huge pages didn't take.  I guess
you'll have to dig into the LINUX documentation a bit to find out what's up
with that.

>>HugePages_Free:      126
>>HugePages_Rsvd:        0
>>HugePages_Surp:        0
>>Hugepagesize:       2048 kB
>>
>>Does tor process automagically take advantage from hugepages after
>>restarting the process or has tor source code to be modified?
>>
>     Olaf, I honestly don't know.  I had not seen the page for which you
>provided a URL, and it is more recent than what I had read about LINUX's
>huge pages before.  Those older articles clearly stated that a program
>had to reserve/designate its memory as huge pages *explicitly*, but it's
>possible that usage is now more automatic.  However, part of the final
>sentence in the article's summary section stands out to me:
>
>	"If your database is running in LINUX *and has HugePages capability*
>	[emphasis mine  --SB], there is no reason not to use it."
>
>That suggests to me that the application (tor, in this case) must tell
>the LINUX kernel which page size it wants for its memory.  Whether it
>also has to specify address ranges explicitly to be so mapped, I haven't
>the foggiest idea.  But even if the application does have to tell the
>kernel something, it ought to be fairly trivial to add to tor's startup
>code.  Start out by overestimating (assuming there is adequate real
>memory on the system to play with) how much tor will need at its maximum
>size, then decrease it, perhaps a bit at a time in successive recompilations,
>until it only minimally exceeds tor's high-water mark.

     BTW, I know that there are *lots* of tor relays running on LINUX
systems whose operators are subscribed to this list.  Don't leave Olaf and
me here swinging in the breeze.  Please jump in with your LINUX expertise
and straighten us out.  Remember that Olaf runs the highest-load-bearing
tor node in our whole network, and there are at least two or four dozen
others that should be considered heavyweight relays that are also on LINUX
systems.  It is in every LINUX+tor user's interest to help Olaf and others
running tor nodes on LINUX systems here to make sure all of those systems
are getting the performance benefits of smaller page tables for their tor
processes (provided those systems have adequate real memory, which I would
bet most of them do).  I've worked with UNIX for decades, but have never
used a LINUX system.  Olaf shouldn't have to depend solely upon someone
who doesn't really know much of what he's writing about to get his blutmagie
tor node running with LINUX huge pages when there are so many LINUX system
experts on this list.

>[soapbox:  on]
>     [Unless you have other applications also using that machine, this would
>probably all be made so much easier by just trying out PC-BSD 8.0 because a
>one-line entry in /boot/loader.conf would take care of superpages for you
>automatically.  PC-BSD is the install-and-go version for both new users who
>need to be able to use the system right away before learning much and casual
>users who have no interest in learning much about FreeBSD.  This special
>packaging of the system is designed to allow both groups, who might
>otherwise find it beyond the effort they were willing or able to put into
>it, to get its benefits.]
>[soapbox:  off]
>     Now that you've had tor running for a while, what does a
>"cat /proc/meminfo | grep -i hugepage" show you?  Also, 126 such pages
>equal 256 MB of memory.  Is that really enough to hold your entire tor
       ^^^ [*252* MB:-]
     Oops.  I guess I should have been heading to bed instead of posting
a followup. :-)

>process when it's going full tilt?  I thought I had seen you post items
>here in the past that said it was taking well over 1 GB and approaching
>2 GB.


                                  Scott Bennett, Comm. ASMELG, CFIAG
**********************************************************************
* Internet:       bennett at cs.niu.edu                              *
*--------------------------------------------------------------------*
* "A well regulated and disciplined militia, is at all times a good  *
* objection to the introduction of that bane of all free governments *
* -- a standing army."                                               *
*    -- Gov. John Hancock, New York Journal, 28 January 1790         *
**********************************************************************
***********************************************************************
To unsubscribe, send an e-mail to majordomo at torproject.org with
unsubscribe or-talk    in the body. http://archives.seul.org/or/talk/