[metrics-bugs] #21236 [Metrics/Metrics website]: Put a visualization of Tor Browser downloads and updates on the Metrics website

[Tor Bug Tracker & Wiki]

#21236: Put a visualization of Tor Browser downloads and updates on the Metrics
website
-------------------------------------+--------------------------------
 Reporter:  karsten                  |          Owner:  karsten
     Type:  enhancement              |         Status:  needs_revision
 Priority:  High                     |      Milestone:
Component:  Metrics/Metrics website  |        Version:
 Severity:  Normal                   |     Resolution:
 Keywords:                           |  Actual Points:
Parent ID:                           |         Points:
 Reviewer:                           |        Sponsor:
-------------------------------------+--------------------------------
Changes (by karsten):

 * status:  merge_ready => needs_revision


Comment:

 I just ran into this bug while importing logs on the Metrics server:

 {{{
      [java] 18:58:10.454 [main] DEBUG org.torproject.metrics.webstats.Main
 - Importing log file
 https://webstats.torproject.org/out/archeotrichon.torproject.org/dist.torproject
 .org-access.log-20151219.xz.
      [java] Exception in thread "main" java.lang.OutOfMemoryError: Java
 heap space
      [java]     at java.util.Arrays.copyOfRange(Arrays.java:2694)
      [java]     at java.lang.String.<init>(String.java:203)
      [java]     at
 java.io.BufferedReader.readLine(BufferedReader.java:349)
      [java]     at
 java.io.BufferedReader.readLine(BufferedReader.java:382)
      [java]     at
 org.torproject.metrics.webstats.Main.downloadLogFile(Main.java:219)
      [java]     at
 org.torproject.metrics.webstats.Main.importLogFiles(Main.java:175)
      [java]     at org.torproject.metrics.webstats.Main.main(Main.java:94)
      [java] Java Result: 1
 }}}

 That log file is 282K compressed and '''1.0G''' uncompressed.  Looks like
 we'll have to try harder not to keep the file in memory at once.  I'm too
 tired to look into this today, but I'll try to fix this tomorrow, in
 particular how we solved this in the [https://gitweb.torproject.org
 /metrics-tasks.git/tree/task-20008/src/Importer.java ad-hoc analysis of
 webstats] in #20008.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21236#comment:16>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online