Banned!

gomby · 03-07-2006, 08:04 AM

I tried out Sunrise XP yesterday to download some sites (well, actually I downloaded their RSS feed). Today my network is banned from Slashdot. Other sites has no problems.

From Slashdot information page:

Either your network or ip address has been banned from Slashdot

...due to script flooding that originated from your network or ip address -- or this IP might have been used to post comments designed to break web browser rendering. Or you crawled us with a rude robot, especially one that doesn't understand RFCs very well.

Can someone inform me about Sunrise XP behavior when crawling a site? Perhaps I made mistakes as well. I tried out configurations to get the best performance ( I use an old computer) and reload each time I change a setting. And Slashdot was the guinea pig for the tweaking.

Jaapjan · 03-07-2006, 08:23 AM

Using a major/large site for 'guinea pigging' is never a good idea, regardless of what you are doing. And if you have no choice, being very careful is always a good thing.

I think slashdot did not like your repeated attempts at crawling the same set of pages? Perhaps you can send them an email and explain what you were trying to do.

gomby · 03-07-2006, 09:00 AM

I chose Slashdot exactly because it is a major site (they will have a big bandwidth, and can withstand some crawling). Well, stupid, perhaps I should try our internal website instead--but then I won't get real-world performance example. I am using my office network.

Actually I was already trying to be careful. I chose "update only if source has changed" Maybe that option didn't help to download.

For information I was trying to find what the best settings for link depth when following links in a site. It seems link depth of 1 is optimal for my computer (and my network).

gomby · 03-07-2006, 09:05 AM

I have just found a post from 2003 that had a similar problem with Slashdot from iSilo section<a href="https://www.mobileread.com/forums/showthread.php?t=330">here </a>. Seems a recurrent problem.

jlogday · 03-07-2006, 09:08 AM

Slashdot has a FAQ about accessing their RSS feeds and what will get you banned:
http://slashdot.org/faq/accounts.shtml#ac1050

Basically, don't hit the RSS more than once every 30 minutes. And yes, even if you use the "update only if source has changed" option, it will still hit the RSS to see if it has changed.

Laurens · 03-07-2006, 09:22 AM

Sunrise XP uses up to two simultaneous connections per server, as recommended by the HTTP spec.

E-mail Slashdot to explain and they'll probably unban you.

Chaos · 03-07-2006, 10:23 AM

I suggest you don't hit Slashdot directly (RSS or otherwise) for reading on a PDA due to it's tendancy to ban. Look at Avantslash, for example.

I've got AvantSlash setup here, and you can use it if you want... Just don't hit it too hard, either, please. (Once, maybe twice a day max, please - it's just an old PII sitting in my basement, after all, with only about 60 kb/s upload. Also you'll probably want to set it to stay on the same server, or else it will pull all the offsite links as well. I could turn the links off, but I figured *someone* might actually want them on.)

gomby · 03-08-2006, 04:05 AM

Quote:

Originally Posted by Laurens

Sunrise XP uses up to two simultaneous connections per server, as recommended by the HTTP spec.

E-mail Slashdot to explain and they'll probably unban you.

Hmm, the problem must be because I load the RSS feed too often then.
Anyway, thanks for all the replies. They are useful. I'll e-mail them soon.

halr9000 · 03-15-2006, 11:45 AM

You may also want to try Alterslash's rss feed.

Note that you'll automatically get unbanned after a period of time, but I don't know what that time is.

03-07-2006, 08:04 AM	#1
gomby Junior Member Posts: 4 Karma: 10 Join Date: Mar 2006 Device: Tungsten E2	Banned! I tried out Sunrise XP yesterday to download some sites (well, actually I downloaded their RSS feed). Today my network is banned from Slashdot. Other sites has no problems. From Slashdot information page: Either your network or ip address has been banned from Slashdot ...due to script flooding that originated from your network or ip address -- or this IP might have been used to post comments designed to break web browser rendering. Or you crawled us with a rude robot, especially one that doesn't understand RFCs very well. Can someone inform me about Sunrise XP behavior when crawling a site? Perhaps I made mistakes as well. I tried out configurations to get the best performance ( I use an old computer) and reload each time I change a setting. And Slashdot was the guinea pig for the tweaking.

03-07-2006, 10:23 AM	#7
Chaos Evangelist Posts: 418 Karma: 281 Join Date: Jul 2004 Location: Canada Device: Assorted older devices	I suggest you don't hit Slashdot directly (RSS or otherwise) for reading on a PDA due to it's tendancy to ban. Look at Avantslash, for example. I've got AvantSlash setup here, and you can use it if you want... Just don't hit it too hard, either, please. (Once, maybe twice a day max, please - it's just an old PII sitting in my basement, after all, with only about 60 kb/s upload. Also you'll probably want to set it to stay on the same server, or else it will pull all the offsite links as well. I could turn the links off, but I figured someone might actually want them on.) Last edited by Chaos; 03-30-2006 at 12:11 AM. Reason: Corrected my avantslash setup url

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Tell Us Your Favorite Banned Book - It's National Banned Books Week	EatingPie	Lounge	96	10-12-2010 02:19 PM
Fagoogle.com - Banned by Google	Colin Dunstan	Lounge	8	09-29-2004 06:42 AM

03-07-2006, 08:23 AM	#2
Jaapjan Avid reader Posts: 262 Karma: 132 Join Date: Mar 2005 Location: The Netherlands Device: HTC Touch Diamond, iLiad Book Edition	Using a major/large site for 'guinea pigging' is never a good idea, regardless of what you are doing. And if you have no choice, being very careful is always a good thing. I think slashdot did not like your repeated attempts at crawling the same set of pages? Perhaps you can send them an email and explain what you were trying to do.

03-07-2006, 09:00 AM	#3
gomby Junior Member Posts: 4 Karma: 10 Join Date: Mar 2006 Device: Tungsten E2	I chose Slashdot exactly because it is a major site (they will have a big bandwidth, and can withstand some crawling). Well, stupid, perhaps I should try our internal website instead--but then I won't get real-world performance example. I am using my office network. Actually I was already trying to be careful. I chose "update only if source has changed" Maybe that option didn't help to download. For information I was trying to find what the best settings for link depth when following links in a site. It seems link depth of 1 is optimal for my computer (and my network).

03-07-2006, 09:05 AM	#4
gomby Junior Member Posts: 4 Karma: 10 Join Date: Mar 2006 Device: Tungsten E2	I have just found a post from 2003 that had a similar problem with Slashdot from iSilo section<a href="https://www.mobileread.com/forums/showthread.php?t=330">here </a>. Seems a recurrent problem.

03-07-2006, 09:08 AM	#5
jlogday Junior Member Posts: 2 Karma: 10 Join Date: Jun 2005 Location: Atlanta, GA Device: Treo 650	Slashdot has a FAQ about accessing their RSS feeds and what will get you banned: http://slashdot.org/faq/accounts.shtml#ac1050 Basically, don't hit the RSS more than once every 30 minutes. And yes, even if you use the "update only if source has changed" option, it will still hit the RSS to see if it has changed.

03-07-2006, 09:22 AM	#6
Laurens Jah Blessed Posts: 1,295 Karma: 1373 Join Date: Apr 2003 Location: The Netherlands Device: iPod Touch	Sunrise XP uses up to two simultaneous connections per server, as recommended by the HTTP spec. E-mail Slashdot to explain and they'll probably unban you.

03-15-2006, 11:45 AM	#9
halr9000 Member Posts: 14 Karma: 12 Join Date: Jan 2005 Device: Sprint/Audiovox PPC-6700	You may also want to try Alterslash's rss feed. Note that you'll automatically get unbanned after a period of time, but I don't know what that time is.

Advert

Advert