[tor-bugs] #20501 [Core Tor/DocTor]: Scan 0.2.9.x relays to see if they're serving an out-of-date consensus?

[Tor Bug Tracker & Wiki]

#20501: Scan 0.2.9.x relays to see if they're serving an out-of-date consensus?
-----------------------------+------------------------
 Reporter:  arma             |          Owner:  atagar
     Type:  enhancement      |         Status:  new
 Priority:  Medium           |      Milestone:
Component:  Core Tor/DocTor  |        Version:
 Severity:  Normal           |     Resolution:
 Keywords:                   |  Actual Points:
Parent ID:                   |         Points:
 Reviewer:                   |        Sponsor:
-----------------------------+------------------------

Comment (by atagar):

 Hi Roger, seems 0.3.0.0 is having issues. 20 of 5002 relays with a DirPort
 are serving stale consensus data. In total there's 22 relays running
 0.3.0.0-alpha-dev so this is just about all of them..

 {{{
 FC64CD763F8C1A319BFBBF62551684F4E1E42332 (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-28T16:00:00
 F864BDEB66C8A11E209456846499C2F995435797 (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-24T07:00:00
 EF887B1997DD786E47CBCAA3F445212467356EBE (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-22T03:00:00
 EF65A4EFF948EA429430FDB834AB9C5F992E81F6 (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-28T16:00:00
 D5B8C38539C509380767D4DE20DE84CF84EE8299 (0.2.9.4-alpha): consensus
 expired at 2016-10-21T16:00:00
 CCC158EA3BB02E4E443BD1F317457CC9E845B050 (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-28T15:00:00
 C79552275DFCD486B942510EF663ED36ACA1A84B (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-27T12:00:00
 B210DF76804619395044D139E826D4AAC939CADB (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-28T15:00:00
 ABF5C38A93F2D7E77A226871AB0ADB052279B48F (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-30T13:00:00
 A6B0521C4C1FB91FB66398AAD523AD773E82E77E (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-28T03:00:00
 A69221A7EC7498D2F88A0FB795261013FA36CAAE (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-26T23:00:00
 9E0B9F902F83E96C352E3B9ECEA4B60B1301A7A1 (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-20T03:00:00
 7FF5DF2E673F4F171C6DB99BA0FA2D9F32A17152 (0.2.9.4-alpha): consensus
 expired at 2016-10-24T22:00:00
 51377C496818552E263583A44C796DF3FB0BC71B (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-28T02:00:00
 48288B0E2473F54A8B9029E0231F2D2F7F7967FD (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-26T22:00:00
 41F07731207742860D43AC426FBAE2F3947BD1CA (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-26T16:00:00
 3DE567C1350C0E858C6147AECB06EA9B3EAF3261 (0.2.9.2-alpha): consensus
 expired at 2016-10-13T07:00:00
 38A42B8D7C0E6346F4A4821617740AEE86EA885B (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-28T02:00:00
 2D49BA84E74E373CD73CDEB09227FDF116155A07 (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-28T15:00:00
 198E72D653B790F7BED7247812C1F96F7CCE56FF (0.3.0.0-alpha-dev): consensus
 expired at 2016-10-27T02:00:00
 }}}

 Full results are attached to this ticket. Here's the script...

 {{{
 import datetime

 from stem.descriptor import DocumentHandler
 from stem.descriptor.remote import get_consensus, get_server_descriptors

 results_file = open('/home/atagar/results.txt', 'w')

 for desc in get_server_descriptors():
   if desc.dir_port:
     results_file.write('%s (%s): ' % (desc.fingerprint, desc.tor_version))
     results_file.flush()

     try:
       consensus = get_consensus(
         endpoints = [(desc.address, desc.dir_port)],
         document_handler = DocumentHandler.BARE_DOCUMENT,
         timeout = 30,
       ).run()[0]
     except Exception as exc:
       results_file.write('failed (%s)\n' % exc)
       continue

     if datetime.datetime.now() > consensus.valid_until:
       results_file.write('consensus expired at %s\n' %
 consensus.valid_until.isoformat())
     else:
       results_file.write('currrent\n')
 }}}

 > Also, once we have a module in our scanners to do this test

 This script took 639 minutes to run (yikes!). Downloading 5002 consensuses
 is a non-trivial load on the network so not something I think we should
 run on an ongoing fashion unless we have reason to suspect a regression.

 > I think all of the relays cache dir stuff, and serve it over their
 ORPort...

 This requires teaching Stem to talk with tor's ORPort (which I'd love!).
 Yawning thought this wouldn't be hard to do.

 https://trac.torproject.org/projects/tor/ticket/18856

 Did you have any other questions?

--
Ticket URL: <https://trac.torproject.org/ticket/20501#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online