impserve: serving content and accessing the internet for the 1200/1150

[ashkulz]

Originally Posted by nrapallo

Nate:

I tried mimicing your set-up and it worked great.

I uninstalled the privoxy software from my system and rebooted. I also changed from using my local IP address for the EBW1150. Now I run impserve using host 127.0.0.1 (and port 9090). I can now surf links WITHIN a .imp ebook, but not for long before I see that 'crash'. Some webpages are probably too much for the EBW1150; that's OK, we'll stay away from them...

Great Can you tell me what kind of pages you were loading? I'd like to isolate the cause so that we can either a) fix up the HTML being served, b) send the HTML in chunks, c) report a bug with the 1150 firmware (they said they'll accept bug reports) -- for the last we'll have to isolate a good test case so that they can reproduce it consistently.

[ashkulz]

BTW, a heads up: I'm in talks with Kovid to integrate ebookutils as plugins into Calibre (A patch to read .IMP and .RB metadata + import into library has already been merged). We still have to work out the details and implement it, so it may take a bit of time (but should get done, eventually).

[nrapallo]

Originally Posted by ashkulz

Great Can you tell me what kind of pages you were loading? I'd like to isolate the cause so that we can either a) fix up the HTML being served, b) send the HTML in chunks, c) report a bug with the 1150 firmware (they said they'll accept bug reports) -- for the last we'll have to isolate a good test case so that they can reproduce it consistently.

I just started with the next.imp ebook for the EBW1150 and choose the third link (test1), fourth (test2), fifth (test3 - sometimes unresponsive), test10 (sometimes resets)...

I think it has to do with local/internal ram being used up and the EBW1150 crashing when a subsequent (unpredictable) page gets served up [which uses up all available ram]. Just pure speculation here; no hard facts!

[nrapallo]

Originally Posted by ashkulz

BTW, a heads up: I'm in talks with Kovid to integrate ebookutils as plugins into Calibre (A patch to read .IMP and .RB metadata + import into library has already been merged). We still have to work out the details and implement it, so it may take a bit of time (but should get done, eventually).

You and Kovid discussed this over a year ago (for PDFRead) and finally we seem to be getting there (for ebookutils)! Good news!

No pressure here, but what about PDFRead being assimulated...

[nrapallo]

I wrote (more like adapted) a plugin for impserve to force webpages to display in the smallest font, where possible.

This could be better tailored to make specific webpages work properly by adjusting some 'class' within a 'div'. Oh, well this is my first go with this idea. In essence, this is a proof-of-concept of a sorts that users can contribute and extend impserve's effectiveness!

Instructions:
=========
1. download the 'AdjustSmallestFont.py.txt' and rename 'AdjustSmallestFont.py'.
2. place the renamed file in your plugins directory (under your ebookutils/impserve directories).
3. run impserve as usual (I issue "run.py -a 127.0.0.1 -p 9090 shelf" from within a .bat file that I double-click. Yes, I'm lazy!)

Enjoy (and contribute back )!

p.s. To modify AdjustSmallestFont.py to increase the xx-small font, change/edit "<font size='1'>" to "<font size='2'>" for x-small font; 3 for small; 4 for medium; 5 for large; and 6 for x-large.

[ashkulz]

Originally Posted by nrapallo

I just started with the next.imp ebook for the EBW1150 and choose the third link (test1), fourth (test2), fifth (test3 - sometimes unresponsive), test10 (sometimes resets)...

I think it has to do with local/internal ram being used up and the EBW1150 crashing when a subsequent (unpredictable) page gets served up [which uses up all available ram]. Just pure speculation here; no hard facts!

Well, I'd appreciate it if someone could quantify it i.e. this site causes problems, or after loading HTML pages of XXX size the ebookwise resets or crashes, etc. Else we will not be able to use the internet browsing feature for the 1150 productively

Originally Posted by nrapallo

You and Kovid discussed this over a year ago (for PDFRead) and finally we seem to be getting there (for ebookutils)! Good news!

No pressure here, but what about PDFRead being assimulated...

Nope, at that point I was discussing integrating the 1100-related features, and this time it covers that and ebookutils. PDFRead was mentioned, but in a kind of "wouldn't it be great..." speculation. Assuming we get the design right, all kinds of conversion tools can be plugged into the pipeline (including PDFRead).

Please note that Calibre is NOT assimulating ebookutils or PDFRead: all we're discussing is a framework for providing plugins to do conversion tasks. Calibre will include some sample tool profiles for ebookutils, rbmake, etc. which can get activated if they're present and call the tools for doing the conversion tasks.

Originally Posted by nrapallo

I wrote (more like adapted) a plugin for impserve to force webpages to display in the smallest font, where possible.

This could be better tailored to make specific webpages work properly by adjusting some 'class' within a 'div'. Oh, well this is my first go with this idea. In essence, this is a proof-of-concept of a sorts that users can contribute and extend impserve's effectiveness!

Great to see you joining the club, and contributing with a plugin! I hope that you can continue with the plugin you mentioned that you were planning to develop

Originally Posted by nrapallo

run impserve as usual (I issue "run.py -a 127.0.0.1 -p 9090 shelf" from within a .bat file that I double-click. Yes, I'm lazy!)

You can directly double click on run.py and it will work, as long as the shelf directory is in the same directory tree -- all the command line switches are optional.

[nrapallo]

Originally Posted by ashkulz

Well, I'd appreciate it if someone could quantify it i.e. this site causes problems, or after loading HTML pages of XXX size the ebookwise resets or crashes, etc. Else we will not be able to use the internet browsing feature for the 1150 productively

It seems to not be that easy to quantify, I mean, if I start with test10 above first its OK, and if I then surf backwards, it will reset by the time I surf back to test2 or test1. The impserve logs don't show anything is wrong per se; the reader just resets. As I said before, this also did happen when I used the Linreb+privoxy method to surf. Perhaps I will try just the PDA-friendly links exclusively ( like www.mobileread.mobi ), and see how far I get.

I don't think this will affect the 'IMP Download Guide', that can now become a reality, thanks to the .imp ebook download 'on-the-fly' ability you coded into impserve.

Nope, at that point I was discussing integrating the 1100-related features, and this time it covers that and ebookutils. PDFRead was mentioned, but in a kind of "wouldn't it be great..." speculation. Assuming we get the design right, all kinds of conversion tools can be plugged into the pipeline (including PDFRead).

I had a vague recollection of "pipelines and such"; thanks for clearing this up for me.

Please note that Calibre is NOT assimulating ebookutils or PDFRead: all we're discussing is a framework for providing plugins to do conversion tasks. Calibre will include some sample tool profiles for ebookutils, rbmake, etc. which can get activated if they're present and call the tools for doing the conversion tasks.

Good to see that no "identity theft" will be involved here...

Great to see you joining the club, and contributing with a plugin! I hope that you can continue with the plugin you mentioned that you were planning to develop

Sorry, which plugin did you volunteer me to write, again? Oh, got to read back a few pages now...

You can directly double click on run.py and it will work, as long as the shelf directory is in the same directory tree -- all the command line switches are optional.

You are even lazier than I !

[ashkulz]

Originally Posted by nrapallo

It seems to not be that easy to quantify, I mean, if I start with test10 above first its OK, and if I then surf backwards, it will reset by the time I surf back to test2 or test1. The impserve logs don't show anything is wrong per se; the reader just resets. As I said before, this also did happen when I used the Linreb+privoxy method to surf. Perhaps I will try just the PDA-friendly links exclusively ( like www.mobileread.mobi ), and see how far I get.

I'll do some testing of my own sometime this weekend (I'll have to the borrow the 1150 which actually belongs to a relative). I don't really like to leave loose ends dangling

Originally Posted by nrapallo

Sorry, which plugin did you volunteer me to write, again? Oh, got to read back a few pages now...

I didn't volunteer you, you came up with the Gutenberg to IMP idea yourself!

[nrapallo]

Originally Posted by ashkulz

I'll do some testing of my own sometime this weekend (I'll have to the borrow the 1150 which actually belongs to a relative). I don't really like to leave loose ends dangling

Using my EBW1150, I managed to surf the www.mobileread.mobi site quite well (50+ 'next' and then go to 'last' page and 10+ 'prev') with no resets in sight! I had my plugin to force the smallest font enabled and it crammed a lot of text onto the screen. I love it!

However, I was getting tired of plain-jane screens so I went to the forums and wiki and lo and behold I managed to get a crash! But I was looking for it to happen... See my log attached below, if it helps!

I didn't volunteer you, you came up with the Gutenberg to IMP idea yourself!

Oh that one! I thought you were referring to this one and my (foolish) acceptance here...

p.s. why does my log say "impmake 0.4.1: starting server on 127.0.0.1:9090" when running impserve...

[Ret]

I'll post the logs I get while trying to surf from nrapallo's test ebook. I try to follow link #9:
impserve:

Loaded plugin: AdjustHtml.py
Loaded plugin: AdjustQueryString.py
impmake 0.4: starting server on 0.0.0.0:9090
127.0.0.1 - - [03/Sep/2008 10:59:06] "GET http://bookshelf.ebooksystem.net/form/redirect.asp?target=http%3A%2F%2F72%2E14%2E205%2E104%2F&pubid=None%3Aebook%3Aguid-237ceaa3f5cb49b281ca33724a38bc5b HTTP/1.0" 302 -
127.0.0.1 - - [03/Sep/2008 10:59:06] "GET http://bookshelf.ebooksystem.net/content/ HTTP/1.0" 200 -

And EBW1150 crashes with error: "Data abort error at SPSR 200000 13 CPSR 20000097....."

While following link #7 (The one pointing to google canada news) I get:

Loaded plugin: AdjustHtml.py
Loaded plugin: AdjustQueryString.py
impmake 0.4: starting server on 0.0.0.0:9090
127.0.0.1 - - [03/Sep/2008 11:04:21] "GET http://bookshelf.ebooksystem.net/form/redirect.asp?target=http%3A%2F%2Fnews%2Egoogle%2Eca%2Fnews%3Fned%3Dtca%26rec%3D0
&pubid=None%3Aebook%3Aguid-237ceaa3f5cb49b281ca33724a38bc5b HTTP/1.0" 302 -
127.0.0.1 - - [03/Sep/2008 11:04:21] "GET http://bookshelf.ebooksystem.net/content/ HTTP/1.0" 200 -

And the crash logs: "Prefetch abort fault at SPSR 200000 13 CPSR 20000097....."
This last link works "correctly" using privoxy.

Does this help?

[Nate the great]

[QUOT=ashkulz;245704]Can you tell me which version worked better (linreb, or an older impserve version)? Also, can you post a log of where it fails so I can improve it [/QUOTE]

I was referring to the versions of the ebook. I will try to get to the log tonight.

Originally Posted by ashkulz

Well, I'd appreciate it if someone could quantify it i.e. this site causes problems, or after loading HTML pages of XXX size the ebookwise resets or crashes, etc. Else we will not be able to use the internet browsing feature for the 1150 productively

I have been testing the link to MobileRead. It doesn't alway work, and I have come up with a few ideas.

a, If the 8MB storage is full or the link is inside a large IMP file, my 1150 will almost certainly crash.

b, If the link is inside a small file, my 1150 will likely not crash.

There could be other causes, but the two situations above have happened consistently.

[nrapallo]

Originally Posted by Ret

I'll post the logs I get while trying to surf from nrapallo's test ebook. I try to follow link #9:
impserve:

Loaded plugin: AdjustHtml.py
Loaded plugin: AdjustQueryString.py
impmake 0.4: starting server on 0.0.0.0:9090
127.0.0.1 - - [03/Sep/2008 10:59:06] "GET http://bookshelf.ebooksystem.net/form/redirect.asp?target=http%3A%2F%2F72%2E14%2E205%2E104%2F&pubid=None%3Aebook%3Aguid-237ceaa3f5cb49b281ca33724a38bc5b HTTP/1.0" 302 -
127.0.0.1 - - [03/Sep/2008 10:59:06] "GET http://bookshelf.ebooksystem.net/content/ HTTP/1.0" 200 -

And EBW1150 crashes with error: "Data abort error at SPSR 200000 13 CPSR 20000097....."

While following link #7 (The one pointing to google canada news) I get:

Loaded plugin: AdjustHtml.py
Loaded plugin: AdjustQueryString.py
impmake 0.4: starting server on 0.0.0.0:9090
127.0.0.1 - - [03/Sep/2008 11:04:21] "GET http://bookshelf.ebooksystem.net/form/redirect.asp?target=http%3A%2F%2Fnews%2Egoogle%2Eca%2Fnews%3Fned%3Dtca%26rec%3D0
&pubid=None%3Aebook%3Aguid-237ceaa3f5cb49b281ca33724a38bc5b HTTP/1.0" 302 -
127.0.0.1 - - [03/Sep/2008 11:04:21] "GET http://bookshelf.ebooksystem.net/content/ HTTP/1.0" 200 -

And the crash logs: "Prefetch abort fault at SPSR 200000 13 CPSR 20000097....."
This last link works "correctly" using privoxy.

Does this help?

I had the same thing happening and to fix it I UNINSTALLED privoxy (and bid it a fond farewell for a job well done! ). I then rebooted and was able to surf the links within a .imp ebook again (but in a limited way ).

[nrapallo]

Originally Posted by Nate the great

a, If the 8MB storage is full or the link is inside a large IMP file, my 1150 will almost certainly crash.

You don't use a 128 MB SM card with the EBW1150? Why 8 MB?

b, If the link is inside a small file, my 1150 will likely not crash.

There could be other causes, but the two situations above have happened consistently.

I've tried the link to Mobileread.com in JSWolf's recent .imp ebooks and got it to work the one or two times I tried. I don't remember how big it was but I have a 128 MB SM card installed with almost 20% free.

[Nate the great]

Originally Posted by nrapallo

You don't use a 128 MB SM card with the EBW1150? Why 8 MB?

Crashing the 1150 seemed to screw up the card. It kept insisting it was half full when no books were on it.

You might want to clear out the 8MB storage on your device, and see if that helps.

[nrapallo]

Originally Posted by Nate the great

Crashing the 1150 seemed to screw up the card. It kept insisting it was half full when no books were on it.

You might want to clear out the 8MB storage on your device, and see if that helps.

Do you know how to format (erase external) the SM card within the ebook reader? You can format it using FAT (not FAT32) with a SM card reader and a PC, but I would still allow the reader to format it (again) before use, just to be sure.

The 8 MB internal storage is there in case there is no SM installed and it would be wise to keep it free of unnecessary files. When the SM card is installed, I think it overrides/supercedes its use.