[tor-bugs] #32133 [Internal Services/Service - git]: gitweb.tpo performance problems

[Tor Bug Tracker & Wiki]

#32133: gitweb.tpo performance problems
---------------------------------------------+--------------------------
 Reporter:  anarcat                          |          Owner:  anarcat
     Type:  defect                           |         Status:  assigned
 Priority:  Immediate                        |      Milestone:
Component:  Internal Services/Service - git  |        Version:
 Severity:  Major                            |     Resolution:
 Keywords:                                   |  Actual Points:
Parent ID:                                   |         Points:
 Reviewer:                                   |        Sponsor:
---------------------------------------------+--------------------------

Comment (by anarcat):

 today we have seen the limits of the cache configuration as a crawler
 trashed through the cache by hitting the random cgit pages it was coming
 from tor so it was not useful to block it by IP either.

 instead I pushed down the MaxRequestWorkers setting from 1500 to 75. that
 seems to have calmed things down. the upstream default is 150, but that's
 still too much for cgit, as 155 cgit processes blows through 8GB of memory
 quite easily.

 in my notes, I had:

 > other possible fix: fcgiwrap or rate-limiting, wait for cache to fill up
 first

 i'm thinking fcgiwrap would be an interesting approach as it would limit
 the number of cgit processes separately from apache.

 right now puppet is on hold on the server to respect the MaxRequestWorkers
 change. it's a puppet-wide setting so i'm hesitant in changing it there.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/32133#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online