[metrics-bugs] #23421 [Metrics/CollecTor]: Use persistence functionality throughout all modules

[Tor Bug Tracker & Wiki]

#23421: Use persistence functionality throughout all modules
-------------------------------+-----------------------------------
 Reporter:  iwakeh             |          Owner:  metrics-team
     Type:  enhancement        |         Status:  needs_information
 Priority:  High               |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:  metrics-2017       |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+-----------------------------------

Comment (by karsten):

 Replying to [comment:7 iwakeh]:
 > [...] Or, do we want to sync invalid descriptors?

 This is an important question!

 Let's assume for a moment that the answer is: yes.

 Right now, the main purpose of having synchronization is to make relay
 descriptor collection more robust by having two separate collectors and an
 easy way to synchronize missing descriptors from one to the other.

 In a way, synchronization is yet another descriptor source, just like
 reading from cached-descriptors files or downloading from directory
 authorities.

 So, maybe we can turn the question around: do we want to handle
 descriptors coming from one source differently than those coming from
 another source?

 If we put the bar higher, I'm worried that we're missing data.

 Imagine that the main instance had a temporary failure and didn't fetch a
 descriptor, but the backup instance fetched it, found it invalid, and
 stored it anyway. If the main instance discards that descriptor during
 synchronization we'll never add it to the archive. And what if we only
 thought it's invalid because of an implementation bug?

 To be honest, I'm only 90% certain that we should treat descriptors from
 those different sources the same, and I'm curious what you think based on
 my comment here.

 Regardless of this specific aspect, I'm all for simplifying things! Thanks
 for pushing this forward!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/23421#comment:8>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online