IP address blocked on certain site

Joe Btfsplk joebtfsplk at gmx.com
Fri Feb 4 21:10:13 UTC 2011


On 2/4/2011 2:42 PM, cmeclax-sazri wrote:
> robots.txt is a file put on some websites as a directive to robots. If 
> you run
> a wiki, and you want only current versions, not the hundreds of previous
> versions of every page, indexed, you could put a directive in robots.txt, or
> label the pages themselves as "noindex nofollow". Automated software that
> ignores such directives is likely to eat up huge amounts of bandwidth and
> create copies that are many times bigger than the original.
>
> cmeclax
So how does that relate to Tor?  Do sites using this see Tor as a 
robot?  I didn't even get on the site.  Plus, there're no files to d/l 
from it - at least not from an avg users capability.
***********************************************************************
To unsubscribe, send an e-mail to majordomo at torproject.org with
unsubscribe or-talk    in the body. http://archives.seul.org/or/talk/



More information about the tor-talk mailing list