Thread: Kindle Paperwhite Jailbreak (5.2.0 - 5.3.1, 5.3.4, 5.3.5)
View Single Post
Old 03-06-2013, 08:26 PM   #391
knc1
Going Viral
knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.knc1 ought to be getting tired of karma fortunes by now.
 
knc1's Avatar
 
Posts: 17,212
Karma: 18210809
Join Date: Feb 2012
Location: Central Texas
Device: No K1, PW2, KV, KOA
Quote:
Originally Posted by ixtab View Post
Hmmm... just found this while looking through the ixtab.tk logs:


72.21.217.98 - - [06/Mar/2013:16:01:43 -0600] "GET /robots.txt HTTP/1.1" 200 52 "-" "Mozilla/5.0 (compatible; AMZNKAssocBot/4.0 +http://affiliate-program.amazon.com)"


Now... Should I be proud, amused, or worried?
Just deny that crawler in that same robots.txt file.
OR
.htaccess them to death
re-direct them to google's db, let them crawl something time consuming.
Last edited by knc1; 03-06-2013 at 08:28 PM.
knc1 is offline   Reply With Quote