[tor-bugs] #16520 [Analysis]: Run some onion services to observe crawling trends

[Tor Bug Tracker & Wiki]

#16520: Run some onion services to observe crawling trends
--------------------------+------------------------------
     Reporter:  arma      |      Owner:
         Type:  project   |     Status:  new
     Priority:  normal    |  Milestone:
    Component:  Analysis  |    Version:
   Resolution:            |   Keywords:  SponsorR, tor-hs
Actual Points:            |  Parent ID:
       Points:            |
--------------------------+------------------------------

Comment (by naif):

 Below a braindump with the morning Espresso.

 The past experience i had with a very simple honeypot (with a shell script
 running from inetd sending me an email) without any active engagement
 highlighted always simple "curl" based crawler, but it took a lot of time
 till the first crawler passed there.

 For intelligence on that i'd suggest to consider two different conditions:
 a) catching crawlers targeting Unpublished TorHS (that we know harvesting
 HSDir)
 b) catching crawlers targeting Published TorHS (that are published on
 Ahmia.fi and/or other Indexes)

 I'd suggest to create tons (thousands) of TorHS every week, focusing on
 automated crawlers with a nice  side-effect of creating many TorHS is that
 the "bad guys" selling the crawled/data will just had some difficulties.
 To do that we must fix #15251 that would also unlock the ability to
 develop "OnionFlare", the Onionized edition of CloudFlare :-)

 For content classification and creation (ie: Anarchy site, Literacy site,
 Drug site, CP site, Political site) i'd suggest to use Ahmia index
 (containing classification) + Tor2web to create reverse-proxy
 Onion<---to--->Onion  (Tor2web do support a "static mapping").

 That way we'll be able to create content and observe timing, behaviours
 and approach in crawling different kind of content without the need to
 create our own, just acting as a "parassitic" network of Onion proxy in
 front of existing Onion proxy.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/16520#comment:7>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online