[tor-bugs] #22738 [Internal Services/Tor Sysadmin Team]: Crawlers slowing down our services

[Tor Bug Tracker & Wiki]

#22738: Crawlers slowing down our services
-----------------------------------------------------+-----------------
     Reporter:  hiro                                 |      Owner:  tpa
         Type:  defect                               |     Status:  new
     Priority:  Medium                               |  Milestone:
    Component:  Internal Services/Tor Sysadmin Team  |    Version:
     Severity:  Normal                               |   Keywords:
Actual Points:                                       |  Parent ID:
       Points:                                       |   Reviewer:
      Sponsor:                                       |
-----------------------------------------------------+-----------------
 Hey,

 Lately crawling is slowing down some of our services. This has been
 affecting trac and gitweb particularly. Maybe we should try to find out if
 requests come from a single crawler and see if we can limit the number of
 requests per amount of time, or just slow it down.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/22738>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online