[tor-bugs] #32231 [Metrics/CollecTor]: Lost some metrics due to broker host migration

[Tor Bug Tracker & Wiki]

#32231: Lost some metrics due to broker host migration
-------------------------------+------------------------------
 Reporter:  cohosh             |          Owner:  metrics-team
     Type:  defect             |         Status:  new
 Priority:  Medium             |      Milestone:
Component:  Metrics/CollecTor  |        Version:
 Severity:  Normal             |     Resolution:
 Keywords:                     |  Actual Points:
Parent ID:                     |         Points:
 Reviewer:                     |        Sponsor:
-------------------------------+------------------------------

Comment (by karsten):

 Replying to [comment:4 cohosh]:
 > Replying to [comment:3 karsten]:
 > > Replying to [comment:2 cohosh]:
 > > > Ideally, we would only collect stats from the broker that people are
 actually using, which right now is the old broker which the freehaven.net
 snowflake domains point to.
 > > >
 > > > Actually, we can probably go ahead and fix this now. How hard would
 it be to point CollecTor to `snowflake-broker at freehaven.net` now and
 replace the metrics files from 2019-10-16 onwards with files from that
 host?
 > >
 > > I just updated CollecTor to point to `snowflake-broker.freehaven.net`
 which means that the new files should show up in a bit.

 Yup, this worked just fine.

 > > The next step would be to delete files we don't need in the archives
 anymore. I'd list them here first before deleting them. I'll do that later
 today or on Monday.
 > Sounds great, thanks!

 Here's the list of files that I'd delete, if you think that's okay:

 {{{
 $ grep -R "snowflake-ips-total" * | sort | grep " [0-9]$"
 2019/10/16/2019-10-16-16-28-15-snowflake-stats:snowflake-ips-total 0
 2019/10/17/2019-10-17-16-28-15-snowflake-stats:snowflake-ips-total 0
 2019/10/18/2019-10-18-19-44-19-snowflake-stats:snowflake-ips-total 1
 2019/10/19/2019-10-19-19-44-19-snowflake-stats:snowflake-ips-total 0
 2019/10/20/2019-10-20-19-44-19-snowflake-stats:snowflake-ips-total 0
 2019/10/21/2019-10-21-19-44-19-snowflake-stats:snowflake-ips-total 1
 2019/10/22/2019-10-22-19-44-19-snowflake-stats:snowflake-ips-total 1
 2019/10/23/2019-10-23-19-44-19-snowflake-stats:snowflake-ips-total 1
 2019/10/24/2019-10-24-19-44-19-snowflake-stats:snowflake-ips-total 0
 2019/10/25/2019-10-25-19-44-19-snowflake-stats:snowflake-ips-total 0
 2019/10/26/2019-10-26-19-44-19-snowflake-stats:snowflake-ips-total 0
 2019/10/27/2019-10-27-19-44-19-snowflake-stats:snowflake-ips-total 0
 2019/10/28/2019-10-28-19-44-19-snowflake-stats:snowflake-ips-total 0
 2019/10/29/2019-10-29-19-44-19-snowflake-stats:snowflake-ips-total 0
 2019/10/30/2019-10-30-19-44-19-snowflake-stats:snowflake-ips-total 0
 2019/10/31/2019-10-31-19-44-19-snowflake-stats:snowflake-ips-total 0
 }}}

 > > > When we finally move the freehaven.net domain to point to the new
 broker, we just need to make sure we don't overwrite these old files again
 (or we could copy over the metrics files to the new host when we do the
 switch).
 > >
 > > I'm not sure I understand what's needed here. Does this require any
 action on the CollecTor side? If so, can you elaborate?
 > No, this doesn't really affect CollecTor, it's more about how we want to
 handle our own copy of snowflake logs. Sorry for the confusion, I'll
 mention that in the discussion of #29258.

 Okay!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/32231#comment:5>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online