Arm Release 1.4.0

Sun Dec 19 11:41:17 UTC 2010

Damian Johnson <atagar1 at gmail.com> wrote:

> > I had to disable the connection panel anyway, as arm kept hogging the CPU
> 
> This is a problem. Arm defaults to making a connection resolution
> every five seconds, and falls back to do them more infrequently if
> they take longer than 50ms. This should mean that the backoff is
> triggered if at least 1% of the total CPU time is spent on connection
> resolutions (handled in lines 344-355 of connections.py [1]). However,
> it sounds like that isn't functioning as intended. :(

I mainly see the problem when scrolling through the connection results:

arm - h942175 (Linux 2.6.26-2-686)         Tor 0.2.2.19-alpha (recommended)
Piper - 81.169.155.246:9001, Dir Port: 9030, Control Port (open): 9051
cpu: 10.3% tor, 92.2% arm  mem: 82 MB (16.4%)  pid: 1774   uptime: 1-18:57:16

I get the impression that the back-off mechanism doesn't apply in
that situation? Otherwise arm tends to stay below 30% cpu usage,
occasionally dropping below 5%.

Another problem I noticed that may or may not be connection related
is that arm occasionally stops working and neither updates the screen
nor reacts to keyboard input.

I previously thought it was caused by a problem with the ssh connection,
but after killing the python process and resetting the terminal the
connection is usable again. Unfortunately I'm not familiar enough
with python to debug this. When the problem happens, python doesn't
seem to use any cpu:

Cpu(s): 11.3%us,  2.0%sy,  0.0%ni, 85.7%id,  0.0%wa,  0.0%hi,  1.0%si,  0.0%st
Mem:    516292k total,   261100k used,   255192k free,    32176k buffers
Swap:  2104504k total,        0k used,  2104504k free,   126948k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 1774 debian-t  20   0 98.2m  84m  19m S 18.3 16.8 276:02.67 tor
 8317 debian-t  20   0  3772 1084  880 S  0.0  0.2   0:00.00 su
 8318 debian-t  20   0  4356 1400  992 S  0.0  0.3   0:00.00 bash
 8319 debian-t  20   0 55296  11m 2724 S  0.0  2.3   2:11.43 python

> Could you please run the test.py script to check the relative runtimes
> for the resolvers? Netstat and sockstat seem to perform the best on
> Linux systems, but if this isn't always the case then I should have
> arm check which is the fastest when picking the default resolver.

h942175:~/arm.git# su -m debian-tor -c 'LANG=; python /root/arm.git/src/test.py'
Arm Test Options:
  1. Resolver Performance Test
  2. Resolver Dump
  3. Glyph Demo
  q. Quit

Selection: 1

----------------------------------------

netstat     478 results     0.0587 seconds
sockstat    478 results     0.2310 seconds
lsof        478 results     0.1100 seconds
Traceback (most recent call last):
  File "/root/arm.git/src/test.py", line 42, in <module>
    connectionResults = connections.getConnections(resolver, "tor", conn.getMyPid())
  File "/root/arm.git/src/util/connections.py", line 135, in getConnections
    results = sysTools.call(cmd)
  File "/root/arm.git/src/util/sysTools.py", line 194, in call
    else: raise errorExc
IOError: 'ss' is unavailable

I didn't find a package for ss, yet.

> Also, did the log message saying "Arm's cpu usage is high (averaging
> X%). You could lower it by dropping the connection data (running as
> "arm -b")." show up?

Yes.

12:40:37 [ARM_WARN] Arm's cpu usage is high (averaging 40.643%). You could lower it by dropping the connection data (running as "arm -b").

> > I had to unset LANG to get the connection resolver working... Probably arm itself should do that.
> 
> This is the first I've heard of this. Do you have any idea what the
> language has to do with connection resolution or arm? I'm weary of
> messing with the environment variables since they're highly system
> dependent and tend to have unintended side effects...

If the output of the resolver utility is (partly) localized
the grep for "ESTABLISHED" no longer matches. However I also
just noticed that only netstat seems to be affected (at least
on the Debian GNU/Linux 5.0 system I'm using).

I think when I first ran into the problem I hadn't installed
sockstat yet and the lsof resolution failed because of the
column mismatch.

> > At least for the lsof 4.78 I'm using, the 8 needs to be a 7.
> 
> Hm, sounds like your lsof version is missing one of the columns.
> Here's what some of my results look like:
> 
> tor       28246 atagar   12u  IPv4 181244      0t0  UDP
> 192.168.0.3:56143->192.168.0.1:53
> tor       28246 atagar   16u  IPv4 181278      0t0  TCP
> 192.168.0.3:47057-><scrubbed>:443 (ESTABLISHED)

Yes, I don't get the "0t0" column:

tor     1774 debian-tor  625u  IPv4 395678       TCP 81.169.155.246:47851-><scrubbed>:9001 (ESTABLISHED)

> I've changed it to strip the "(ESTABLISHED)" part and use the last
> column so it should be happy with both of our versions.

It works, thanks.

> > ... and an improperly tested hack to show the external address between the local and the foreign one, if the local and the external one differ.
> 
> I tried applying this one (with a few fixes) but the connection panel
> isn't great code to hack up. This introduces issues with column
> alignment, confusing controller entries like:
> 127.0.0.1:9051  -->  89.188.20.246  -->  127.0.0.1:55224
> and probably a few other headaches for edge cases like small screen resolutions.

Hmm, I thought the confusing controller case should be handled
with the ipAddressIsPrivate() patch (it works for me), but I
agree that there are a bunch of other problems with the patch.

> The problem isn't really the change, but rather that this part of the
> arm codebase kinda sucks. I'm not sure if you noticed, but the
> connection panel and controller differ from all the rest of the
> components. The reason is that arm's been going through a rewrite and
> these are the last bits of the old codebase left.
> 
> I like this change, and the connection panel is the next part due for
> a rewrite, so while I'm revising it I'll be sure to include this.

Great, thanks.

Fabian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: not available
URL: <http://lists.torproject.org/pipermail/tor-dev/attachments/20101219/f1c39b80/attachment.pgp>