[tor-bugs] #20335 [Metrics/CollecTor]: ReferenceChecker causes OOM
Tor Bug Tracker & Wiki
blackhole at torproject.org
Thu Oct 13 13:05:51 UTC 2016
#20335: ReferenceChecker causes OOM
-------------------------------+---------------------
Reporter: iwakeh | Owner:
Type: defect | Status: new
Priority: Medium | Milestone:
Component: Metrics/CollecTor | Version:
Severity: Normal | Resolution:
Keywords: | Actual Points:
Parent ID: | Points:
Reviewer: | Sponsor:
-------------------------------+---------------------
Comment (by karsten):
Replying to [comment:4 iwakeh]:
> Steps to reproduce (from #18910, the discussion should take place here):
>
[https://gitweb.torproject.org/user/iwakeh/collector.git/commit/?h=task-18910
-first-sync&id=5eb89c10a0a720c1fa4eb806e5d5b5f2dee75b65 Collector branch]:
Run the jar in a new location, i.e. with empty recent and out. Configure
relaydescs and sync. RunOnce will be sufficient the OOM heap dump contains
very many Reference objects. (Your assumption in one of the previous
comments hinted in the right direction, I think, but I couldn't verify
that yet.)
Hmm, I didn't run out of memory while checking references. And I don't
quite understand how that would happen in an initial run, because the
reference checker is run before the sync manager even starts to run.
I did run out of heap space while reading descriptors in a second run:
{{{
$ java -DLOGBASE=log -jar collector-1.1.0-dev.jar
2016-10-13 12:54:55,995 ERROR o.t.d.i.DescriptorReaderImpl:192 Bug:
uncaught exception or error while reading descriptors: Java heap space
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:3236) ~[na:1.8.0_102]
at
java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
~[na:1.8.0_102]
at
java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)
~[na:1.8.0_102]
at
java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:153)
~[na:1.8.0_102]
at
org.torproject.descriptor.impl.DescriptorReaderImpl$DescriptorReaderRunnable.readFile(DescriptorReaderImpl.java:378)
~[collector-1.1.0-dev.jar:1.1.0-dev-5eb89c1]
at
org.torproject.descriptor.impl.DescriptorReaderImpl$DescriptorReaderRunnable.readDescriptors(DescriptorReaderImpl.java:284)
~[collector-1.1.0-dev.jar:1.1.0-dev-5eb89c1]
at
org.torproject.descriptor.impl.DescriptorReaderImpl$DescriptorReaderRunnable.run(DescriptorReaderImpl.java:188)
~[collector-1.1.0-dev.jar:1.1.0-dev-5eb89c1]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
}}}
Can you try to put the reference checker back in and see if you still run
out of memory somewhere? I'm not yet convinced that it's the reference
checker that is to blame here.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20335#comment:7>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
More information about the tor-bugs
mailing list