View Single Post
Old 02-04-2009, 10:18 AM   #1
tungsten
Junior Member
tungsten began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Feb 2009
Device: none
Question How to slow down the download/spidering?

I'm trying to download (just once) a page that doesn't like to be downloaded.
Therefore I get loads of "too many accesses per second" errors.

How can I prevent this?
I have two suggestions:
(1) Slow down the download so something like 1 page per second. How would you do this?
(2) I can split it up into maybe 50 threads. In that case, I would have to set up a "master page" which links all these in one file. How can you do that?

Sorry to post this in an archive-thread, but where should I post this?
tungsten is offline