View Single Post
Old 09-23-2009, 11:24 AM   #4
Wolferl
Junior Member
Wolferl began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Sep 2009
Device: Compaq iPAQ
Hi Laurens,

Quote:
Originally Posted by Laurens View Post
No, there is no setting to configure spidering speed. You might want to limit the max. number of connections to get around this.
You mean "View -> Preferences -> Network tab -> "maximum active updates"?
I already set this to "1". Unfortunately, Sunrise runs two requests at the same time then. There seems to be an "off by one" bug...

But anyways, that does not really solve the problem:
Klack-channels.de seems to block access with a "403 forbidden" if one retrieves pages too quickly.
I tried to use a special proxy software that allows throtteling (Charles Web Debugging Proxy), and using this, it was possible to get the whole thing spidered without any errors. It just takes 20 minutes (only 600kb file size)...and that proxy is not freeware.

If there would be configurable delay "delay before request" in seconds, that would be perfect.

Cheers,
Wolferl
Wolferl is offline