[tor-bugs] #20335 [Metrics/CollecTor]: ReferenceChecker causes OOM

[Tor Bug Tracker & Wiki]

#20335: ReferenceChecker causes OOM
-------------------------------+---------------------
 Reporter:  iwakeh             |          Owner:
     Type:  defect             |         Status:  new
 Priority:  Medium             |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:                     |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+---------------------

Comment (by karsten):

 Replying to [ticket:20335 iwakeh]:
 > In general, statistics and analysis should be separated from normal
 functioning.

 Separated how?  We need to run these checks every hour to learn early if
 we're missing more descriptors than usual.

 > After a sync-run ReferenceChecker hogs CPU and memory.  It might need to
 be disabled for the first sync-release.

 Do you have an idea why it would do that after a sync run and not after a
 usual download run?  I'd rather not want to disable the checker in a
 release and instead fix it.  I can try to take a look if you give me
 something to reproduce the issue.

 > The implementation should be redesigned.

 How?  Maybe we can get away by fixing it for the moment.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20335#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online