[tor-bugs] #2954 [Tor Directory Authority]: ides corrupted its cached-microdescs.new file

[Tor Bug Tracker & Wiki]

#2954: ides corrupted its cached-microdescs.new file
----------------------------------------+-----------------------------------
    Reporter:  mikeperry                |       Owner:  nickm             
        Type:  defect                   |      Status:  assigned          
    Priority:  normal                   |   Milestone:  Tor: 0.2.3.x-final
   Component:  Tor Directory Authority  |     Version:                    
  Resolution:                           |    Keywords:                    
      Parent:                           |      Points:                    
Actualpoints:                           |  
----------------------------------------+-----------------------------------
Changes (by rransom):

  * priority:  critical => normal


Comment:

 Replying to [comment:5 rransom]:
 > Notice the `@last-listed` dates -- ides had been corrupting its
 microdesc cache for over a year, but didn't OOM in the process of trying
 to parse the entire tail of its MD cache until this month, when the file
 had become ''much'' longer.

 Again, I speculated rather than RTFSing...  It didn't try to parse the
 entire tail of the MD cache, just the text until just before the next line
 that begins with “`@`” or “`onion-key`”.

 > > Inspecting the microdesc cache revealed that several microdescs
 appeared to be just running into the next without proper termination,
 perhaps a side effect of earlier crashes/ooms.
 >
 > `microdescs_add_list_to_cache` and `dump_microdescriptor` are scary.
 Perhaps we should be prefixing each item in the `cached-*.new` files with
 a line containing the cached item's length and a short (32 or fewer bits)
 hash, and trying to resynchronize if we read a damaged item.

 This doesn't seem to be necessary in order to prevent a runaway parser,
 but it would help us recover more items from the cache when something does
 go wrong.

 Decreasing priority, because I am now convinced that the cache corruption
 didn't increase ides's memory consumption significantly.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/2954#comment:7>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online