|
But imagine someone requesting the feed every 5 mins. I had a few users who were polling the PalmGear feeds from our site every 5 seconds (!!!). Even worse, their feed client didn't support conditional gets, so it fetched the entire feed each time. Nor did it support gzipping. So every 5 seconds the user polled both PalmGear feeds which are each around 20k uncompressed. In other words, every minute he plugged almost half a Mb of bandwidth from us.
This is I think Colin's major fear with scraping sites like wotzwot. Do they re-scrape the target site every time a feed is polled (perhaps by several people even), or do you they localled cache the reformatted feeds. When I have some more time, I will give it a try and use it on some MobileRead resource, so that i can check our log files later to see how wotzwot handles this issue.
|