[tor-bugs] #32473 [Metrics/Exit Scanner]: Evaluate the results from the exitmap based scanner compared to the current exit lists system

[Tor Bug Tracker & Wiki]

#32473: Evaluate the results from the exitmap based scanner compared to the current
exit lists system
----------------------------------+------------------------------
 Reporter:  irl                   |          Owner:  metrics-team
     Type:  task                  |         Status:  new
 Priority:  Medium                |      Milestone:
Component:  Metrics/Exit Scanner  |        Version:
 Severity:  Normal                |     Resolution:
 Keywords:                        |  Actual Points:
Parent ID:                        |         Points:
 Reviewer:                        |        Sponsor:
----------------------------------+------------------------------

Comment (by karsten):

 I compared those exit lists from the new exit scanner (only 2019-11-26 or
 later) to the ones published by the old exit scanner from the same time
 frame. I attached my output file that organizes scan results by
 fingerprint. Some observations:

  - The new scanner omits a handful relays that the old scanner scanned. I
 assume this is related to exit policies? The absolute number of these
 cases is rather low, but please take a look, too (at the beginning of the
 output file).

  - The vast majority of scans (over 1,000) result in the same exit
 address. There are just 6 relays with 2 or more exit addresses by either
 old or new scanner. It's hard to say which scanner is right here. Maybe
 it's just relays on dynamic IPs or with multiple IP addresses. Please take
 a look at these (grep for "exit addresses total").

  - Looks like the new scanner doesn't properly clear old scan results. I'm
 only looking at results from 2019-11-26 and later, and yet there are scan
 results from 2019-11-19 and 2019-11-20. This seems like a (small) bug.

 That's all. Overall, I think that these results are as close to the
 current scanner as they can get. If we can resolve that one bug and get
 your confirmation that the other two things are normal, I think we're good
 to go.

 And glad to hear that you have found a way for including published/last
 status times!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/32473#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online