[tor-bugs] #27836 [Internal Services/Service - trac]: RSS feed https authentication

[Tor Bug Tracker & Wiki]

#27836: RSS feed https authentication
--------------------------------------------------+-----------------
     Reporter:  atagar                            |      Owner:  qbi
         Type:  defect                            |     Status:  new
     Priority:  Medium                            |  Milestone:
    Component:  Internal Services/Service - trac  |    Version:
     Severity:  Normal                            |   Keywords:
Actual Points:                                    |  Parent ID:
       Points:                                    |   Reviewer:
      Sponsor:                                    |
--------------------------------------------------+-----------------
 Hi trac admins. I maintain the r2e feed that provides our tor-wiki-
 changes@ list [1]. Its been broken for a while and today I spent a few
 hours digging into why.

 This list relies on trac's RSS feed [2]. Https looks to be broken on it.
 In particular...

 * **Firefox** can view the feed, but unlike other trac pages indicates the
 site is insecure ('Page Info > Security' says the page is unencrypted and
 lacks any certificate).

 * **Curl** responds with a 403...

 {{{
 % curl -s
 'https://trac.torproject.org/projects/tor/timeline?wiki=on&format=rss' |
 grep 'Forbidden'
       Error: Forbidden – Tor Bug Tracker & Wiki
           <h1>Error: Forbidden</h1>
 }}}

 * **Python's urllib** (which is what r2e uses) fails with a 403 as well.
 It can access other trac pages...

 {{{
 import urllib.request
 request = urllib.request.Request('https://trac.torproject.org/')
 }}}

 ... but the feed fails with...

 {{{
 import urllib.request
 request =
 urllib.request.Request('https://trac.torproject.org/projects/tor/timeline?wiki=on&format=rss')
 }}}

 {{{
 % python3 scrap.py
 Traceback (most recent call last):
   File "scrap.py", line 4, in <module>
     urllib.request.urlopen(request)
   File "/usr/lib/python3.6/urllib/request.py", line 223, in urlopen
     return opener.open(url, data, timeout)
   File "/usr/lib/python3.6/urllib/request.py", line 532, in open
     response = meth(req, response)
   File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
     'http', request, response, code, msg, hdrs)
   File "/usr/lib/python3.6/urllib/request.py", line 570, in error
     return self._call_chain(*args)
   File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
     result = func(*args)
   File "/usr/lib/python3.6/urllib/request.py", line 650, in
 http_error_default
     raise HTTPError(req.full_url, code, msg, hdrs, fp)
 urllib.error.HTTPError: HTTP Error 403: Forbidden
 }}}

 At this point I'm pretty stumped. Something's broken with the feed and I'm
 unsure how to work around it. We don't care about encryption on these
 requests so if providing a plain http feed is easier than fixing this then
 happy to go with that too (presently http is a 302 to https).

 Thanks!

 [1] https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-wiki-changes
 [2] https://trac.torproject.org/projects/tor/timeline?wiki=on&format=rss

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/27836>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online