[metrics-bugs] #27356 [Metrics/ExoneraTor]: Reduce database size and variance of query response times

[Tor Bug Tracker & Wiki]

#27356: Reduce database size and variance of query response times
--------------------------------+------------------------------
 Reporter:  karsten             |          Owner:  metrics-team
     Type:  enhancement         |         Status:  merge_ready
 Priority:  High                |      Milestone:
Component:  Metrics/ExoneraTor  |        Version:
 Severity:  Normal              |     Resolution:
 Keywords:                      |  Actual Points:
Parent ID:                      |         Points:
 Reviewer:  irl                 |        Sponsor:
--------------------------------+------------------------------

Comment (by karsten):

 Replying to [comment:14 irl]:
 > Replying to [comment:13 karsten]:
 > > Do they also have an automatically combined version with all changes
 under source control, or do they otherwise explain how to obtain the
 latest schema? If this is obvious for their contributors, maybe it's also
 obvious for our future contributors.
 >
 > Well, you can run all the upgrade scripts in sequence without running
 the importer and then export the result. I don't think they have an
 annotated SQL file, but separate documentation.

 Okay.

 Regarding version information in the database, I took a less invasive way
 that keeps the current migration process as simple as possible: the
 `exonerator2.sql` script now exits on first error to avoid multiple
 executions, cf.
 [https://gitweb.torproject.org/user/karsten/exonerator.git/commit/?h=task-27356&id=b64f4d1c1320c1f43d1e5dc67549897b2e2e8e66
 commit b64f4d1]. In the longer term we'll need something more elaborate,
 but maybe we can design something that then works for all databases,
 including those used in metrics-web.

 Still working on migration/deployment.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/27356#comment:15>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online