[metrics-bugs] #21551 [Metrics/Metrics website]: Merge Metrics sub sites into main Tor Metrics website

[Tor Bug Tracker & Wiki]

#21551: Merge Metrics sub sites into main Tor Metrics website
-------------------------------------+------------------------------
 Reporter:  karsten                  |          Owner:  metrics-team
     Type:  enhancement              |         Status:  new
 Priority:  High                     |      Milestone:
Component:  Metrics/Metrics website  |        Version:
 Severity:  Normal                   |     Resolution:
 Keywords:                           |  Actual Points:
Parent ID:                           |         Points:
 Reviewer:                           |        Sponsor:
-------------------------------------+------------------------------

Comment (by iwakeh):

 Replying to [comment:3 karsten]:
 > Agreed on moving
 [https://trac.torproject.org/projects/tor/ticket/21551#comment:1 comment 1
 above] to #19754.
 >
 > But regarding the original description, not moving forward with this
 ticket is becoming a liability. We're running
 [https://gitweb.torproject.org/karsten/metrics-web.git/log/?h=staging4 my
 staging4 branch] on the Tor Metrics website for a few months now, because
 we haven't merged the `collector.html` and `onionoo.html` pages into
 [https://gitweb.torproject.org/metrics-web.git/log/ master] yet. I'm
 cherry-picking new commits from the master branch and sometimes resolving
 conflicts while doing so. I don't expect it to be much fun to rebase my
 staging4 branch to master, and it'll become even less fun over time.
 >
 > Another issue is that I'm avoiding making changes to the CollecTor and
 Onionoo pages, because we'd currently have to make them in two places: in
 their respective repos and in my metrics-web staging4 branch. As a result,
 the typos fixed in #22169 are only fixed on
 https://onionoo.torproject.org/, but not on
 https://metrics.torproject.org/onionoo.html. And I'm holding back the
 #22263 change, because I want to avoid applying that patch twice.
 >
 > In short, let's make some decisions here and implement them. Open
 points, and my suggestions, are:

 Yes, this situation should be resolved swiftly:

 >
 >  - How should we support the use case of mirror operators running
 modified versions of Onionoo or CollecTor and wanting to reflect that by
 providing a modified specification page?
 >    - Our original plan was to keep specifications in the Onionoo or
 CollecTor repositories and only include them in metrics-web's `.war` file
 using a Git submodule and some Ant magic. This turns out to be non-
 trivial. While the bulk of pages stays the same, there are a lot of subtle
 differences.
 >    - ''My suggestion:'' How about we kill the `.html` pages in the
 Onionoo and CollecTor repositories and consider the pages in metrics-web
 as new masters? After all, Onionoo and CollecTor are Metrics subprojects
 and the Tor Metrics website is the primary website for Metrics projects.
 And if mirror operators want to adapt these pages, they can quite easily
 `wget` and update them as needed. I feel we're otherwise optimizing for a
 quite special use case here, making the main use case of providing
 specifications for Metrics services unnecessarily hard.

 I agree that removing the website html-pages and adding them to Metrics-
 Web is the best option here.  (I would debate considering Onionoo and
 CollecTor 'sub-projects', but this doesn't hinder applying the given
 solution.)  Metrics-Web provides documentation and protocol definitions
 for Tor Metrics.

 >    - In the future we could consider providing a schema file on both
 Onionoo and Collector which metrics-web fetches and uses to generate a
 human-readable specification page. This is possibly even a fun project,
 but it's not one that should delay moving away from staging4 much longer.

 Hmm, I'd just keep the doc/definition files in one place, which can be
 Metrics-Web.  Just wait and see if this topic comes up again?

 >  - Should we let users browser CollecTor files on the Tor Metrics
 website rather than in Apache-generated directory listings on the
 CollecTor page?
 >    - ''My suggestion:'' I'd say yes, but that can be done in a
 subsequent patch. Also a fun project.

 Yes, new ticket!

 >  - How do we include `.onion` links on the placeholder pages on Onionoo
 and CollecTor?
 >    - We briefly speculated about possible ways to detect how the user
 accesses the placeholder page, and only show the onion or non-onion link.
 >    - ''My suggestion:'' For now, I'd say let's just show both.

 I vote for showing both:  sometimes one needs the plain-web link and
 browses using tor and also the other way around.  No need to hide any
 links.

 >  - How do we support the case of different Onionoo hosts running
 different Onionoo protocol versions?
 >    - We said we'd include something like "added in version X" and
 "removed in version Y" in the specification. Sounds totally doable, but
 maybe in the near future as a subsequent patch.
 >    - ''My suggestion:'' For now, let's just show the latest protocol
 version specification, because there's really just one Onionoo protocol
 version deployed.

 Agreed.  Let's keep it simple.

 >
 > If these suggestions sound reasonable, I'll rebase my staging4 branch to
 master, include the #22169 and #22263 fixes, and put it here for review.
 And then we'll create tickets for the suggested follow-up tasks (schema
 files for Onionoo and CollecTor, browse CollecTor files based on
 `index.json`, create placeholder pages with onion and direct link to Tor
 Metrics, include information when an Onionoo document or field was
 added/removed) and do them in the near future as time permits.
 >
 > Thoughts, ideas, things that I overlooked? Thanks!

 All added above.  Good to move this forward!

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21551#comment:4>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online