[tor-relays] Circuit creation "storms" overwhelming > Raspberry Pi?
temp5 at tormail.org
temp5 at tormail.org
Tue Jul 9 19:46:52 UTC 2013
FWIW, I did some tuning on mine (not a pi, but had similar issues) and it
seems more stable now too. I added the following to my torrc (where X and
Y are your rates/units):
MaxOnionsPending 250
BandwidthRate X
BandwidthBurst Y
RelayBandwidthRate X
RelayBandwidthBurst Y
I had the RelayBandwidth options already in it, but I read somewhere the
added Bandwidth options above should limit directory port traffic bursts.
And MaxOnionsPending allows more to queue up before creating errors.
It sounds like the next major version of Tor (in release candidate stage
right now) has several performance improvements, so you might just want to
wait for that to release and see if the problems disappear on their own
thanks to the hard work of the developers!
> From: Gordon Morehouse <gordon at morehouse.me>
>
> Hi, Yes. This is absolutely on my to-do list. I've had a family
> medical emergency and about 2 or 3 other things recently about that
> level of stress, but BELIEVE me, a strategy for getting a Raspberry Pi
> to be a rock solid relay is of paramount importance to me.
>
> I'm hoping to figure out all the tweaks I did, and then make an alpha
> version of some iptables rules to defend against the "storms," within
> the next couple of weeks.
>
> Please don't hesitate to bug me about this at my direct email. (The old
> one is no longer valid, I've dropped pseudonyms on the tor lists.)
>
> Thanks much,
> -Gordon M.
>
>
> Thomas Hand:
>> Hi torsion,
>>
>> I'm also running a tor relay on a raspberry pi and keep getting these
>> storm
>> creation events which crash the box. You said you made some adjustments
>> to
>> the configs to get a more stable system? Can you please email me copies
>> of
>> the configs or maybe list the changes you made?
>> I'm trying to get my pi stable and then perhaps role out a few more
>> relay
>> nodes to friends and colleagues.
>> Thanks,
>>
>> T
>>
>>
>> On 4 May 2013 22:07, <torsion at ftml.net> wrote:
>>
>>> I did a lot of tuning on the Raspberry Pi and it's now much, much more
>>> stable as a Tor relay, but just now I had another "circuit creation
>>> storm." Interestingly, the Pi remained up, and my *router* crashed.
>>> I've also seen huge bursts of circuit creation on a relay I run on a
>>> VPS, but as it's a much more powerful box it rarely complains (and thus
>>> I rarely notice).
>>>
>>> This many circuits and outbound connections is highly unusual for the
>>> small relay I'm running on the Pi. And this behavior definitely occurs
>>> in bursts. Is this an outbound DDOS or an attack on Tor itself? If
>>> the
>>> former (or maybe the latter), is there some way I could perhaps use
>>> iptables to temporarily "clamp" the ability to open TCP connections
>>> when
>>> Tor (or anything on the Pi) opens a number over some threshold in some
>>> short period of time?
>>>
>>> Here's log output (via 'arm') from the relay after my router crashed
>>> twice, I went to the admin panel and noted hundreds of outbound
>>> connections from my Tor box. Time is America/Los_Angeles.
>>>
>>> ? 13:55:00 [ARM_NOTICE] Relay unresponsive (last heartbeat: Sat May 4
>>> 13:54:14 2013)
>>> ? 13:52:25 [WARN] Your computer is too slow to handle this many
>>> circuit
>>> creation
>>> ? requests! Please consider using the MaxAdvertisedBandwidth config
>>> option or choosing
>>> ? a more restricted exit policy. [404 similar message(s) suppressed
>>> in last 60 seconds]
>>> ? 13:51:07 [WARN] Your computer is too slow to handle this many
>>> circuit
>>> creation
>>> ? requests! Please consider using the MaxAdvertisedBandwidth config
>>> option or choosing
>>> ? a more restricted exit policy. [75 similar message(s) suppressed
>>> in
>>> last 60 seconds]
>>> ? 13:50:52 [WARN] Your computer is too slow to handle this many
>>> circuit
>>> creation
>>> ? requests! Please consider using the MaxAdvertisedBandwidth config
>>> option or choosing
>>> ? a more restricted exit policy. [601 similar message(s) suppressed
>>> in last 60 seconds]
>>> ? 13:48:39 [WARN] Your computer is too slow to handle this many
>>> circuit
>>> creation
>>> ? requests! Please consider using the MaxAdvertisedBandwidth config
>>> option or choosing
>>> ? a more restricted exit policy. [99 similar message(s) suppressed
>>> in
>>> last 60 seconds]
>>> ? 13:47:34 [WARN] Your computer is too slow to handle this many
>>> circuit
>>> creation
>>> ? requests! Please consider using the MaxAdvertisedBandwidth config
>>> option or choosing
>>> ? a more restricted exit policy. [22 similar message(s) suppressed
>>> in
>>> last 60 seconds]
>>> ? 13:46:17 [WARN] Your computer is too slow to handle this many
>>> circuit
>>> creation
>>> ? requests! Please consider using the MaxAdvertisedBandwidth config
>>> option or choosing
>>> ? a more restricted exit policy. [253 similar message(s) suppressed
>>> in last 60 seconds]
>>> ? 13:43:47 [WARN] Your computer is too slow to handle this many
>>> circuit
>>> creation
>>> ? requests! Please consider using the MaxAdvertisedBandwidth config
>>> option or choosing
>>> ? a more restricted exit policy. [1396 similar message(s) suppressed
>>> in last 60
>>> ? seconds]
>>> ? 13:42:48 [WARN] Your computer is too slow to handle this many
>>> circuit
>>> creation
>>> ? requests! Please consider using the MaxAdvertisedBandwidth config
>>> option or choosing
>>> ? a more restricted exit policy. [16 similar message(s) suppressed
>>> in
>>> last 60 seconds]
>>>
>>> Here's how it crashed my router (blowing ip_conntrack limits is
>>> sufficient only to mess up many of my TCP connections, but eventually
>>> the router runs out of memory and starts killing processes):
>>>
>>> May 4 13:51:24 dedmaus user.warn kernel: ip_conntrack: table full,
>>> dropping packet.
>>> May 4 13:51:24 dedmaus user.warn kernel: ip_conntrack: table full,
>>> dropping packet.
>>> May 4 13:51:24 dedmaus user.warn kernel: ip_conntrack: table full,
>>> dropping packet.
>>> May 4 13:51:25 dedmaus user.warn kernel: ip_conntrack: table full,
>>> dropping packet.
>>> May 4 13:51:29 dedmaus user.warn kernel: NET: 152 messages suppressed.
>>> May 4 13:51:29 dedmaus user.warn kernel: ip_conntrack: table full,
>>> dropping packet.
>>> May 4 13:51:34 dedmaus user.warn kernel: NET: 193 messages suppressed.
>>> May 4 13:51:34 dedmaus user.warn kernel: ip_conntrack: table full,
>>> dropping packet.
>>> May 4 13:51:39 dedmaus user.warn kernel: NET: 227 messages suppressed.
>>>
>>> ...ad infinitum with the number of messages suppressed per 5 sec
>>> increasing until the router crashes.
>>>
>>>
>>>
>>> On Mon, Mar 18, 2013, at 06:18 PM, torsion at ftml.net wrote:
>>>> I'm also seeing occasional messages like this on the Pi (it never
>>>> lasts
>>>> long):
>>>>
>>>> 18:13:24 [ARM_NOTICE] Relay resumed
>>>> 18:13:18 [ARM_NOTICE] Relay unresponsive (last heartbeat: Mon Mar 18
>>>> 18:13:04 2013)
>>>> 17:28:43 [ARM_NOTICE] Relay resumed
>>>> 17:28:38 [ARM_NOTICE] Relay unresponsive (last heartbeat: Mon Mar 18
>>>> 17:28:25 2013)
>>>> 14:12:26 [ARM_NOTICE] Relay resumed
>>>> 14:12:20 [ARM_WARN] Deduplication took too long. Its current
>>>> implementation has difficulty handling large logs so disabling it to
>>>> keep the interface responsive.
>>>> 14:12:20 [ARM_NOTICE] Relay unresponsive (last heartbeat: Mon Mar 18
>>>> 14:12:06 20
>>>>
>>>> On Mon, Mar 18, 2013, at 01:00 PM, torsion at ftml.net wrote:
>>>>> Hi there, I just joined the mailing list and apologized if this has
>>> been
>>>>> discussed before. I did find discussion of a similar issue in
>>>>> January
>>>>> 2013's archive:
>>>>>
>>>>>
>>> https://lists.torproject.org/pipermail/tor-relays/2013-January/001809.html
>>>>>
>>>>> It's important to note that I believe I've seen (but didn't save
>>>>> logs)
>>> a
>>>>> couple "circuit creation burst" events on my established relay (about
>>>>> 5Mbps, stable, guard, non-exit) which was mostly able to handle it
>>>>> without crashing as it has plenty of RAM and the above-mentioned
>>>>> messages - "Your computer is too slow to handle this many circuit
>>>>> creation requests! Please consider using the MaxAdvertisedBandwidth
>>>>> config option or choosing a m ore restricted exit policy." - appear
>>> only
>>>>> with the relay is under load for other reasons AND a large number of
>>>>> circuits are being suddenly created.
>>>>>
>>>>> I wondered if this was some kind of DOS attempt but didn't think much
>>> of
>>>>> it because my fast relay continued working fine.
>>>>>
>>>>> However, I've just set up a Raspberry Pi, the 512MB model, as a relay
>>> on
>>>>> a slower connection. Here are the relevant settings on this relay:
>>>>>
>>>>> RelayBandwidthRate 130 KB
>>>>> RelayBandwidthBurst 340 KB
>>>>>
>>>>> The Pi has a fairly slow CPU, so I'd occasionally get messages about
>>> log
>>>>> deduplication being too slow or something, but didn't think much of
>>>>> it.
>>>>> I finally got the relay up and left it up for over 24 hours. When I
>>>>> woke up this morning it had crashed. Here are the relevant log
>>> messages
>>>>> - note the huge jump in number of circuits between 22:35 and 04:35
>>>>> (maybe I got the Stable flag), then the storm of circuit open
>>>>> requests
>>>>> starting at 05:49. Eventually I believe the Pi ran out of memory and
>>>>> killed the tor process.
>>>>>
>>>>> What's very interesting here is that my fast VPS relay with a
>>>>> RelayBandwidthRate over 5x faster is almost never handling much more
>>>>> than 1000 circuits, so why all of a sudden the demand on the Pi
>>>>> that's
>>>>> advertising a lower bandwidth rate?
>>>>>
>>>>> Mar 17 22:35:00.000 [notice] Heartbeat: Tor's uptime is 1 day 0:00
>>>>> hours, with 26 circuits open. I've sent 974.13 MB and received 969.92
>>>>> MB.
>>>>> Mar 18 04:35:00.000 [notice] Heartbeat: Tor's uptime is 1 day 6:00
>>>>> hours, with 972 circuits open. I've sent 1.61 GB and received 1.59
>>>>> GB.
>>>>> Mar 18 05:49:44.000 [warn] Your computer is too slow to handle this
>>> many
>>>>> circuit creation requests! Please consider using the
>>>>> MaxAdvertisedBandwidth config option or choosing a more restricted
>>>>> exit
>>>>> policy.
>>>>> Mar 18 05:49:44.000 [warn] Failed to hand off onionskin. Closing.
>>>>> Mar 18 05:50:44.000 [warn] Your computer is too slow to handle this
>>> many
>>>>> circuit creation requests! Please consider using the
>>>>> MaxAdvertisedBandwidth config option or choosing a more restricted
>>>>> exit
>>>>> policy. [5817 similar message(s) suppressed in last 60 seconds]
>>>>> Mar 18 05:52:30.000 [warn] Your system clock just jumped 101 seconds
>>>>> forward; assuming established circuits no longer work.
>>>>> Mar 18 05:53:51.000 [warn] Your computer is too slow to handle this
>>> many
>>>>> circuit creation requests! Please consider using the
>>>>> MaxAdvertisedBandwidth config option or choosing a more restricted
>>>>> exit
>>>>> policy. [1055 similar message(s) suppressed in last 60 seconds]
>>>>> Mar 18 05:55:14.000 [warn] Your computer is too slow to handle this
>>> many
>>>>> circuit creation requests! Please consider using the
>>>>> MaxAdvertisedBandwidth config option or choosing a more restricted
>>>>> exit
>>>>> policy. [329 similar message(s) suppressed in last 60 seconds]
>>>>>
>>>>> I'd like to figure out just how much the Raspberry Pi is capable of,
>>>>> because it could be a cheap way to build out the relay network by
>>> people
>>>>> who want to donate bandwidth - but of course it needs to be stable,
>>>>> and
>>>>> something about my setup is not.
>>>>>
>>>>> Also:
>>>>>
>>>>> Mar 16 20:55:33.000 [notice] No AES engine found; using AES_*
>>> functions.
>>>>>
>>>>> I have no idea if the Broadcom BCM2835 SoC (ARM1176JZF-S CPU) in the
>>>>> Pi
>>>>> has any AES capability, but it'd be great to find out.
>>>>>
>>>>> _______________________________________________
>>>>> tor-relays mailing list
>>>>> tor-relays at lists.torproject.org
>>>>> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
>>> _______________________________________________
>>> tor-relays mailing list
>>> tor-relays at lists.torproject.org
>>> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
>>>
>>
>>
>>
>> _______________________________________________
>> tor-relays mailing list
>> tor-relays at lists.torproject.org
>> https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
>>
>
>
More information about the tor-relays
mailing list