Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > Miscellaneous > Archive > Sunrise

Notices

 
 
Thread Tools Search this Thread
Old 09-22-2009, 07:27 AM   #1
Wolferl
Junior Member
Wolferl began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Sep 2009
Device: Compaq iPAQ
Question Update size is different every attempt?

Hi folks,

I am just new to SunriseXP (having read the FAQ )
After AvantGo shut down, I wanted to have another possibility to have a TV guide on my Compaq iPAQ (PPC, Vade Mecum reader). I tried Plucker, but it is not able to handle cookies, which is important for www.klack-channels.de. So the search continued...
SunriseXP did the job well at first sight. Then I discovered, that some links were not downloaded. I increased link depth and retried. Now, even less stuff is created! The pdb file, originally 220kb, is now between 2k and 50kb, it changes on every "update" event I do.

Question 1:
How come it does not update all the stuff?

Question 2:
Is there a setting "maximum file size" or something which prevents SunriseXP downloading ALL the stuff?

I'm a bit lost...

Thanks for help,
Wolferl
Wolferl is offline  
Old 09-23-2009, 06:43 AM   #2
Wolferl
Junior Member
Wolferl began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Sep 2009
Device: Compaq iPAQ
I've done a bit of research and found out that SunriseXP gets errors when retrieving some of the HTML files (notHandled: 403 Forbidden).

It seems there is a problem requesting the files too quickly.

Is there a setting to slow spidering down a bit? Or re-fetching the HTML if it fails?

Cheers,
Wolfgang
Wolferl is offline  
Advert
Old 09-23-2009, 09:13 AM   #3
Laurens
Jah Blessed
Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.
 
Laurens's Avatar
 
Posts: 1,295
Karma: 1373
Join Date: Apr 2003
Location: The Netherlands
Device: iPod Touch
No, there is no setting to configure spidering speed. You might want to limit the max. number of connections to get around this. Unfortunately, this is a global setting that affects all sites.
Laurens is offline  
Old 09-23-2009, 10:24 AM   #4
Wolferl
Junior Member
Wolferl began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Sep 2009
Device: Compaq iPAQ
Hi Laurens,

Quote:
Originally Posted by Laurens View Post
No, there is no setting to configure spidering speed. You might want to limit the max. number of connections to get around this.
You mean "View -> Preferences -> Network tab -> "maximum active updates"?
I already set this to "1". Unfortunately, Sunrise runs two requests at the same time then. There seems to be an "off by one" bug...

But anyways, that does not really solve the problem:
Klack-channels.de seems to block access with a "403 forbidden" if one retrieves pages too quickly.
I tried to use a special proxy software that allows throtteling (Charles Web Debugging Proxy), and using this, it was possible to get the whole thing spidered without any errors. It just takes 20 minutes (only 600kb file size)...and that proxy is not freeware.

If there would be configurable delay "delay before request" in seconds, that would be perfect.

Cheers,
Wolferl
Wolferl is offline  
Old 09-23-2009, 04:27 PM   #5
Laurens
Jah Blessed
Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.
 
Laurens's Avatar
 
Posts: 1,295
Karma: 1373
Join Date: Apr 2003
Location: The Netherlands
Device: iPod Touch
There's not much I can do for you, unfortunately, since I'm no longer maintaining the application. You might want to look around for some other (hopefully free) proxy that can handles bandwidth throttling.
Laurens is offline  
Advert
Old 09-24-2009, 04:32 AM   #6
Wolferl
Junior Member
Wolferl began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Sep 2009
Device: Compaq iPAQ
Hi Laurens,

Quote:
Originally Posted by Laurens View Post
There's not much I can do for you, unfortunately, since I'm no longer maintaining the application.
Since I have lots of programming experience...what do you think about doing this modification myself? I have searched around for the source, but wasn't successful...

Cheers,
Wolferl
Wolferl is offline  
Old 09-25-2009, 08:43 AM   #7
Laurens
Jah Blessed
Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.Laurens is no ebook tyro.
 
Laurens's Avatar
 
Posts: 1,295
Karma: 1373
Join Date: Apr 2003
Location: The Netherlands
Device: iPod Touch
Quote:
Originally Posted by Wolferl View Post
Hi Laurens,



Since I have lots of programming experience...what do you think about doing this modification myself? I have searched around for the source, but wasn't successful...

Cheers,
Wolferl
SunriseXP (C++) is unfortunately not open-source. It uses some closed-source libraries, preventing its release as an open-source project.

You might want to consider switching to the older Java version, which is open-source.

http://sourceforge.net/projects/sunrisexp/
Laurens is offline  
Old 09-26-2009, 05:04 AM   #8
Wolferl
Junior Member
Wolferl began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Sep 2009
Device: Compaq iPAQ
Hi Laurens,

Quote:
Originally Posted by Laurens View Post
SunriseXP (C++) is unfortunately not open-source. It uses some closed-source libraries, preventing its release as an open-source project.
Oh, too bad

Quote:
Originally Posted by Laurens View Post
You might want to consider switching to the older Java version, which is open-source.
http://sourceforge.net/projects/sunrisexp/
OK, I'll stick with this one.

Thanks for your help!

Wolferl
Wolferl is offline  
 

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Attempt to update JBL - REDUX steven522 Ectaco jetBook 1 08-24-2010 10:57 PM
anyone else toughing out font size while waiting for firmware update? BBel Kobo Reader 33 06-27-2010 10:08 PM
Evince update attempt tjzeeman iRex 5 02-07-2010 01:03 PM
My first attempt at writing an original... JanG Writers' Corner 13 08-11-2009 05:36 PM
My first upload attempt lilac_jive Workshop 1 01-17-2009 10:46 PM


All times are GMT -4. The time now is 03:57 PM.


MobileRead.com is a privately owned, operated and funded community.