[tor-bugs] #1641 [Metrics]: Make website logs available in the Metrics Portal

[Tor Bug Tracker & Wiki]

#1641: Make website logs available in the Metrics Portal
---------------------+------------------------------------------------------
 Reporter:  karsten  |       Owner:  karsten 
     Type:  task     |      Status:  assigned
 Priority:  minor    |   Milestone:          
Component:  Metrics  |     Version:          
 Keywords:           |      Points:          
   Parent:           |  
---------------------+------------------------------------------------------

Comment(by phobos):

 A few random thoughts.  In discussions with the EFF, they recommended this
 plan from a legal perspective, https://www.eff.org/wp/osp.  This suggests
 we could log in the default CLF and purge the logs within 48 hours.  This
 plan would save space and let us see summarized results.

 I'd like to be able to see:

  1. visits and unique visitor counts by days of the week, day of the month
  1. top 25 pages by page view
  1. country of origin
  1. most viewed, entry, and exit pages
  1. downloads by package
  1. http errors
  1. referrers (sanitized if it includes PII)
  1. search engines, keyphrases and keywords
  1. overall traffic by webserver, if possible (is our load balancing
 working evenly?)
  1. top 10 paths through the site
  1. number of pages viewed per visit
  1. percentage and count of requests through tor exits



 Or another idea is to see what we can provide from the feature list
 of http://awstats.sourceforge.net/ without disclosing PII.

 And we should remember that this is more than just the logs for www.tpo,
 we have check, svn, gitweb, metrics, bridges, and trac websites to
 analyze.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/1641#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online