Quote:
|
Originally Posted by hacker
What did these "big sites" have to say about your users, through the use of your software, depriving them of their advertiser revenue by bypassing their banner ads and other advertising resources when deep-linking to their print-only pages?
|
None of them complained. Statistically, Sunrise does not make an impact.
Last time I checked, Plucker Desktop came configured with global exclusion filters for well-known ad URLs (windcaster and such). Did no-one complain about their lost ad revenue?
Quote:
|
Originally Posted by hacker
Incidentally, your tool (v0.41f) completely ignores robots.txt, which makes adhering to Entity Tags, Last-Modified, Cache-Control, and so on... basically irrelevant as far as bandwidth savings go. Your tool blindly allows slamming a site for content as fast as possible over and over and over until it has it all, or exhausts maximum fetch depth.
|
Plucker and iSilo also ignore robots.txt, don't they? Now why is this a problem all of a sudden? And how does ignoring robots.txt make caching irrelevant?