[tor-dev] Spinning off Directory Archive from Metrics Portal

[Karsten Loesing]

Hello devs,

I'm continuously tweaking the Metrics Portal [0] in the attempt to make
it more useful.  My latest idea is to finally spin off the Directory
Archive part from it, which is the part that serves descriptor tarballs.
 I'd like to hear what people think about that.

Let me give you some more context.  The Metrics Portal serves three main
purposes:

 1. Graphs [1]: there are graphs on network size, network diversity,
user number estimates, and performance measurements.  This is probably
what most visitors are interested in.  In addition to graphs, there are
also tables and .csv files for download.

 2. Research [2]: we offer descriptor tarballs for download and explain
the data formats.  This is mostly interesting for researchers and
developers.  (This is the part that I'd like to spin off and move to a
separate place.)

 3. Status [3]: we provided (and still provide) services that are based
on archived descriptors.  This includes ExoneraTor and Consensus Health,
both of which have moved to their own sites, and it includes Relay
Search which is high on the list of endangered services.  While we (did)
provide these services on the Metrics Portal, they're almost gone.

So, my plan is to move away the Research part, number 2 above, and
re-organize the remaining Graphs part to address visitors who are not
necessarily researchers or developers.  The new Directory Archive
website would contain the following content:

 - Start page: possibly re-using parts from the current start page [0].
 - Data: How to obtain the data, possibly re-using parts from the
current Data page [4], but without separate pages for file lists.
 - Formats: similar to the current Data Formats page [5].
 - Tools: similar to the current Tools page [6].

What would be a good name for the new website holding the Tor Directory
Archive?  How about:

 - CollecTor, collector.torproject.org (not available yet) or
 - AggregaTor, aggregator.torproject.org (not available yet)

On the software side, I'd like to remove all dynamic (Java) parts from
the new website and have it served by Apache alone instead of Tomcat.
The only parts that still need to be dynamically generated would be file
lists, and I'd like to solve that by using Apache directory listings or
some other Apache module.

The re-organized Metrics Portal is going to have the following content:

 - Start page: possibly re-using parts from the current start page [0].
 - Graphs: all four sub pages from the current Graphs page [1], so
Network, Bubbles, Users, and Performance.
 - Aggregated data: the processed data behind graphs that is currently
available on the Statistics page [7].

Speaking of, if anybody wants to help design the new website (or help
re-design the existing Metrics Portal website once the cruft is gone),
your help would be much appreciated.  Bonus points if no JavaScript is
required, at least for the Directory Archive website.  Please contact me
if you're interested.

Any feedback welcome!  Thanks!

All the best,
Karsten


[0] https://metrics.torproject.org/
[1] https://metrics.torproject.org/graphs.html
[2] https://metrics.torproject.org/research.html
[3] https://metrics.torproject.org/status.html
[4] https://metrics.torproject.org/data.html
[5] https://metrics.torproject.org/formats.html
[6] https://metrics.torproject.org/tools.html
[7] https://metrics.torproject.org/stats.html