[tor-bugs] #2895 [BridgeDB]: BridgeDB assumes that cached-descriptors[.new] are in chronological order

[Tor Bug Tracker & Wiki]

#2895: BridgeDB assumes that cached-descriptors[.new] are in chronological order
--------------------------+----------------------------------------------
     Reporter:  karsten   |      Owner:  isis
         Type:  defect    |     Status:  closed
     Priority:  minor     |  Milestone:
    Component:  BridgeDB  |    Version:
   Resolution:  fixed     |   Keywords:  bridgedb-parsers, bridgedb-0.3.0
Actual Points:            |  Parent ID:
       Points:            |
--------------------------+----------------------------------------------
Changes (by isis):

 * status:  accepted => closed
 * keywords:   => bridgedb-parsers, bridgedb-0.3.0
 * resolution:   => fixed


Comment:

 If this was referring to the `cached-extrainfo` and `cached-extrainfo.new`
 files (to my knowledge, BridgeDB has never had `cached-descriptor*`
 files), then this is a bug (related to #11216), and it would mean that the
 `transport` lines of newer descriptors would potentially be overwritten by
 older, duplicate descriptors.

 If that's the bug we're talking about, then here's the fix. :) Otherwise,
 feel free to reopen and/or add more information.

 This was fixed by commits
 [https://gitweb.torproject.org/user/isis/bridgedb.git/commit/?h=fix/9380-stem_r10&id=7869e4c7cd1e43f9354480f6cafba0794fd86433
 7869e4c7cd1e43f9354480f6cafba0794fd86433],
 [https://gitweb.torproject.org/user/isis/bridgedb.git/commit/?h=fix/9380-stem_r10&id=65f18cd7c31a97274a8277688f0512b69ac1e3e3
 65f18cd7c31a97274a8277688f0512b69ac1e3e3], and
 [https://gitweb.torproject.org/user/isis/bridgedb.git/commit/?h=fix/9380-stem_r10&id=fe70415269693948bdfc5c8ea3abfab2b1d86c49
 fe70415269693948bdfc5c8ea3abfab2b1d86c49] in my `fix/9380-stem_r10`
 [https://gitweb.torproject.org/user/isis/bridgedb.git/log/?h=fix/9380-stem_r10
 branch].

 Those commits introduce the `bridgedb.parse.descriptors.deduplicate()`
 [https://gitweb.torproject.org/user/isis/bridgedb.git/tree/lib/bridgedb/parse/descriptors.py?h=fix/9380-stem_r10#n149
 function], which is called in the
 `bridgedb.parse.descriptors.parseExtraInfoFiles()`
 [https://gitweb.torproject.org/user/isis/bridgedb.git/tree/lib/bridgedb/parse/descriptors.py?h=fix/9380-stem_r10#n219
 function]. The former deduplicates all descriptors for every bridge,
 selecting only the newest descriptor for a particular bridge.
 Additionally, if any Bridge has multiple `@type bridge-extrainfo`
 descriptors with exactly the same timestamps, then a
 `bridgedb.parse.descriptors.DescriptorWarning`
 [https://gitweb.torproject.org/user/isis/bridgedb.git/tree/lib/bridgedb/test/test_parse_descriptors.py?h=fix/9380-stem_r10#n371
 will be issued], since perfectly identical descriptors shouldn't be
 something an unmodified tor is capable of doing (and thus would imply that
 there is either a drastic regression in tor, or that someone has created a
 possibly-malicious OR implementation). Unittests and integration
 [https://gitweb.torproject.org/user/isis/bridgedb.git/tree/lib/bridgedb/test/test_parse_descriptors.py?h=fix/9380-stem_r10#n381
 tests] which verify that these behaviours are functioning as expected have
 also been added.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/2895#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online