Hi Laurens,
Quote:
Originally Posted by Laurens
No, there is no setting to configure spidering speed. You might want to limit the max. number of connections to get around this.
|
You mean "View -> Preferences -> Network tab -> "maximum active updates"?
I already set this to "1". Unfortunately, Sunrise runs two requests at the same time then. There seems to be an "off by one" bug...
But anyways, that does not really solve the problem:
Klack-channels.de seems to block access with a "403 forbidden" if one retrieves pages too quickly.
I tried to use a special proxy software that allows throtteling (Charles Web Debugging Proxy), and using this, it was possible to get the whole thing spidered without any errors. It just takes 20 minutes (only 600kb file size)...and that proxy is not freeware.
If there would be configurable delay "delay before request" in seconds, that would be perfect.
Cheers,
Wolferl