[metrics-bugs] #29461 [Metrics/CollecTor]: Add a Snowflake module

[Tor Bug Tracker & Wiki]

#29461: Add a Snowflake module
-------------------------------------------------+-------------------------
 Reporter:  irl                                  |          Owner:
                                                 |  metrics-team
     Type:  enhancement                          |         Status:
                                                 |  needs_revision
 Priority:  Medium                               |      Milestone:
Component:  Metrics/CollecTor                    |        Version:
 Severity:  Normal                               |     Resolution:
 Keywords:  metrics-roadmap-august, anti-        |  Actual Points:
  censorship-roadmap-september                   |
Parent ID:                                       |         Points:  8
 Reviewer:  irl                                  |        Sponsor:
                                                 |  Sponsor28
-------------------------------------------------+-------------------------

Comment (by karsten):

 Replying to [comment:12 cohosh]:
 > Replying to [comment:9 karsten]:
 > Thanks karsten!
 > >  - As indicated before, I'm using `snowflake-stats-end` as descriptor
 type identifier, which means that future data format versions will have to
 keep that line as their first line. A better choice would have been to use
 something like `snowflake-stats $version` or similar. (If it's any relief,
 we're forced to use `[0-9]{10}` as descriptor type identifier for
 bandwidth files, so there would have been plenty of room to do worse.)
 > Oh this is a good point. I can make this change pretty easily now.

 Okay, in that case please make this change and let me know where to find
 the new spec and a new sample.

 > >  - The current format only supports a single snowflake broker. Maybe
 this is acceptable for the snowflake design. But just in case that you'll
 one day want to add a second broker, you'll have to include some sort of
 broker identifier in the format.
 > This will definitely be the case for quite a while, I think a spec
 change (and version bump of the spec) will be a good way to handle this.
 How difficult will it be to have data with different spec versions on your
 end?

 Not difficult at all. Okay, let's keep this unchanged then.

 > >  - The current format is not signed, which is somewhat related to not
 having a broker identifier in the format.
 > >  - As a consequence of the above, CollecTor needs to make a decision
 whether it wants to archive a newly downloaded snowflake-stats snippet, if
 it already has another snippet with the same timestamp and different
 contents. Possible strategies for this specific case are to a) never
 overwrite, b) always overwrite, c) keep all versions by including a digest
 in the file name, d) maybe something else. I implemented a) for now.
 > I didn't think about signing, that would take while to implement I
 think. I agree that the best decision here is (a) for now.
 > >
 > > I think we can start with what we have, without changing anything of
 the above. Of course, if you want to change something with regard to
 future maintenance effort, now's the time!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/29461#comment:16>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online