03-07-2006, 08:04 AM | #1 |
Junior Member
Posts: 4
Karma: 10
Join Date: Mar 2006
Device: Tungsten E2
|
Banned!
I tried out Sunrise XP yesterday to download some sites (well, actually I downloaded their RSS feed). Today my network is banned from Slashdot. Other sites has no problems.
From Slashdot information page: Either your network or ip address has been banned from Slashdot ...due to script flooding that originated from your network or ip address -- or this IP might have been used to post comments designed to break web browser rendering. Or you crawled us with a rude robot, especially one that doesn't understand RFCs very well. Can someone inform me about Sunrise XP behavior when crawling a site? Perhaps I made mistakes as well. I tried out configurations to get the best performance ( I use an old computer) and reload each time I change a setting. And Slashdot was the guinea pig for the tweaking. |
03-07-2006, 08:23 AM | #2 |
Avid reader
Posts: 262
Karma: 132
Join Date: Mar 2005
Location: The Netherlands
Device: HTC Touch Diamond, iLiad Book Edition
|
Using a major/large site for 'guinea pigging' is never a good idea, regardless of what you are doing. And if you have no choice, being very careful is always a good thing.
I think slashdot did not like your repeated attempts at crawling the same set of pages? Perhaps you can send them an email and explain what you were trying to do. |
03-07-2006, 09:00 AM | #3 |
Junior Member
Posts: 4
Karma: 10
Join Date: Mar 2006
Device: Tungsten E2
|
I chose Slashdot exactly because it is a major site (they will have a big bandwidth, and can withstand some crawling). Well, stupid, perhaps I should try our internal website instead--but then I won't get real-world performance example. I am using my office network.
Actually I was already trying to be careful. I chose "update only if source has changed" Maybe that option didn't help to download. For information I was trying to find what the best settings for link depth when following links in a site. It seems link depth of 1 is optimal for my computer (and my network). |
03-07-2006, 09:05 AM | #4 |
Junior Member
Posts: 4
Karma: 10
Join Date: Mar 2006
Device: Tungsten E2
|
I have just found a post from 2003 that had a similar problem with Slashdot from iSilo section<a href="https://www.mobileread.com/forums/showthread.php?t=330">here </a>. Seems a recurrent problem.
|
03-07-2006, 09:08 AM | #5 |
Junior Member
Posts: 2
Karma: 10
Join Date: Jun 2005
Location: Atlanta, GA
Device: Treo 650
|
Slashdot has a FAQ about accessing their RSS feeds and what will get you banned:
http://slashdot.org/faq/accounts.shtml#ac1050 Basically, don't hit the RSS more than once every 30 minutes. And yes, even if you use the "update only if source has changed" option, it will still hit the RSS to see if it has changed. |
03-07-2006, 09:22 AM | #6 |
Jah Blessed
Posts: 1,295
Karma: 1373
Join Date: Apr 2003
Location: The Netherlands
Device: iPod Touch
|
Sunrise XP uses up to two simultaneous connections per server, as recommended by the HTTP spec.
E-mail Slashdot to explain and they'll probably unban you. |
03-07-2006, 10:23 AM | #7 |
Evangelist
Posts: 418
Karma: 281
Join Date: Jul 2004
Location: Canada
Device: Assorted older devices
|
I suggest you don't hit Slashdot directly (RSS or otherwise) for reading on a PDA due to it's tendancy to ban. Look at Avantslash, for example.
I've got AvantSlash setup here, and you can use it if you want... Just don't hit it too hard, either, please. (Once, maybe twice a day max, please - it's just an old PII sitting in my basement, after all, with only about 60 kb/s upload. Also you'll probably want to set it to stay on the same server, or else it will pull all the offsite links as well. I could turn the links off, but I figured *someone* might actually want them on.) Last edited by Chaos; 03-30-2006 at 12:11 AM. Reason: Corrected my avantslash setup url |
03-08-2006, 04:05 AM | #8 | |
Junior Member
Posts: 4
Karma: 10
Join Date: Mar 2006
Device: Tungsten E2
|
Quote:
Anyway, thanks for all the replies. They are useful. I'll e-mail them soon. Last edited by gomby; 03-08-2006 at 04:08 AM. Reason: grammar |
|
03-15-2006, 11:45 AM | #9 |
Member
Posts: 14
Karma: 12
Join Date: Jan 2005
Device: Sprint/Audiovox PPC-6700
|
You may also want to try Alterslash's rss feed.
Note that you'll automatically get unbanned after a period of time, but I don't know what that time is. |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Tell Us Your Favorite Banned Book - It's National Banned Books Week | EatingPie | Lounge | 96 | 10-12-2010 02:19 PM |
Fagoogle.com - Banned by Google | Colin Dunstan | Lounge | 8 | 09-29-2004 06:42 AM |