impserve: serving content and accessing the internet for the 1200/1150 [Archive]

Ret

08-29-2008, 09:41 AM

IMPRESSIVE work! Congratulations ashkulz! Thank you for your effort!
I downloaded Portable Python and in a minute Impserve was up and running!

I'd change the first hard-coded web page to use an index.html file. It nows shows two links: local contents, and online library which I find to be unnecessary. Just open local contents page.

I'm adding the HTML code I use as index.html to show a Google search, a link to an IMP book in mobileread and a link to the forum message where it was found.
So far it seems you cannot directly download ebooks from Mobileread. As I said before, this is becausem Mobileread webservers (as almost ALL webservers) doesn't know the MIME type application/x-softbook and handles IMP files with type unknown/unknown. Is there a way to change those HTTP headers to "Conten-type: applicaction/x-sofbook"?

Thanks again for your code. It's incredible that such few lines of code resolves HOURS of testing, settings, etc. Well done!

<html>

<head>
<title>Welcome to ImpServe</title>
</head>

<body>
<p><h1> Welcome to Impserve</h1>
</p>
<p> </p>
<p> </p>

<center>
<img alt="Google" src="http://www.google.com/intl/en/images/about_logo.gif" height=65 width=175>
</center>
<p>
<form name=Search method=GET action="http://www.google.com/search?hl=en">
<input type=text name='q' size=22 maxlength=2048 value="">
<input type=submit name="btnG" value="Search"></form>
</p>
<p> </p>
<center>
<a href="http://www.mobileread.com/forums/attachment.php?attachmentid=15610&d=1219722410
">Link to an IMP ebook in Mobileread</a>
<a href="http://www.mobileread.com/forums/showthread.php?t=28232">Link to a Mobileread forum</a>
</center>

</body>

</html>

nrapallo

08-29-2008, 10:03 AM

IMPRESSIVE work! Congratulations ashkulz! Thank you for your effort! ... Thanks again for your code. It's incredible that such few lines of code resolves HOURS of testing, settings, etc. Well done!

I echo the same sentiment. You look at that python code and say "WOW". I guess you have to be a techie to really appreciate what Ashish has done!

Thanks, Ret, for the startup page; it was a bit minimalist. I would leave the 'Online Bookshelf' link, for nothing else, but the novelty of it!

It only gets better and better... What next, whispernet! :rofl:

ashkulz

08-29-2008, 10:31 AM

I have tried the same ebook reader actions under two 'surf the internet' solutions: one being impserve; the other being 'Linreb+privoxy'. I only used the REB1200, so far, haven't tried the EBW1150 yet.

I attach the log results from both and you can see that the transfer times are much slower in impserve. Now to find the cause of it and if it is system related...

BTW, Ashish, notice the last log entry; it's a delete request from the online bookshelf that could be implemented in impserve. Linreb also ignores that delete request. Food for thought... I know the problem, it's that when the log is printed the it does a reverse lookup on the IP address of your ebook, which is what causes the latency. I fixed it by adding an entry to the hosts file yesterday, but forgot to add the fix: I'm attaching 0.2.1 for now. I'll be working on a 0.3 release for tomorrow, so please let the feature requests come in -- I'd really appreciate a comprehensive list so I know what to shoot for :)

nrapallo

08-29-2008, 11:18 AM

please let the feature requests come in -- I'd really appreciate a comprehensive list so I know what to shoot for :)

OK, first on my list would be, as stated before:

Cache "Content-type: text/html" transfers and before you send them onto the ebook reader, in response to the HTTP GET, text substitute, at least:
- <a href> WITH <a UNDERLINE="YES" href> (so that you can SEE hyperlinks)
- <body> WITH <body style="margin-left:2%; margin-right:2%"> (so there's no edge-bleeding - can use 2px instead of 2% or even 0)
In addition, to simulate a back function, remember the last html "visited" and when the directory icon is tapped ( HTTP requests to directory.ebooksystem.net, I think ) then go there. It would be nice to remember ALL sites visited so that we could go back to the start page, if we liked!

However, I see where this could pose a problem; if we want a quick restart. I would recommend that a 'bookshelf' request then clear this cached list of websites visited and restore the directory icon's function. What do you think?

That's it for now...

Did I mention whispernet? :rofl:

Ret

08-29-2008, 03:54 PM

Don't forget to:
- Handle internet links within ebooks. Impserve should be able of stripping the redirection URL that ebooksystem.net uses.
- Change content-type header to application/x-softbook when downloading an IMP file from a server that doesn't know that MIME type.
- Let the reader delete ebooks from the online bookshelf.

that's all that comes to mind.... THANKS!

nrapallo

08-29-2008, 08:34 PM

I know the problem, it's that when the log is printed the it does a reverse lookup on the IP address of your ebook, which is what causes the latency. I fixed it by adding an entry to the hosts file yesterday, but forgot to add the fix: I'm attaching 0.2.1 for now. I'll be working on a 0.3 release for tomorrow, so please let the feature requests come in -- I'd really appreciate a comprehensive list so I know what to shoot for :)

WOW, that returned and maybe improved my surfing speed. No more slow downs with WinXP and impserve!!!

Truly amazing!

I can't wait to see your version 0.3... :drool: :drool: (wipes chin now) :drool: :drool:

p.s. Ashish, I attach a privoxy log showing three ways to get a .imp ebook from Mobileread.com; each with a different Content-type! Please see attached text log.

This may help determine the best way to download .imp ebooks directly from Mobileread!

ashkulz

08-30-2008, 04:31 AM

WOW, that returned and maybe improved my surfing speed. No more slow downs with WinXP and impserve!!!

Truly amazing!

I can't wait to see your version 0.3... :drool: :drool: (wipes chin now) :drool: :drool:

p.s. Ashish, I attach a privoxy log showing three ways to get a .imp ebook from Mobileread.com; each with a different Content-type! Please see attached text log.

This may help determine the best way to download .imp ebooks directly from Mobileread! I've already figured out #2 and #3: look at the "Content-disposition" header, and if it provides a filename call mimetypes.guess_type (http://docs.python.org/lib/module-mimetypes.html) on the file which will give the correct MIME type. I'm more hesitant of doing it for URLs where they don't specify Content-disposition and the URL ends with .imp and the content-type is not correct: it may be a valid URL. You can choose to override it in the plugins system I plan to develop.

nrapallo

08-30-2008, 06:33 AM

I've already figured out #2 and #3: look at the "Content-disposition" header, and if it provides a filename call mimetypes.guess_type (http://docs.python.org/lib/module-mimetypes.html) on the file which will give the correct MIME type. I'm more hesitant of doing it for URLs where they don't specify Content-disposition and the URL ends with .imp and the content-type is not correct: it may be a valid URL. You can choose to override it in the plugins system I plan to develop.

I think Example #3, in my privoxy log file above, would be the most common, as it will come about when surfing the IMP ebook upload forum threads and selecting the .imp attachment therein (doesn't end with .imp).

Example #2 is what would be expected if the "IMP Download Guide" could be prepared by Alexander in .imp format and/or .html format similar to the current 'Mobipocket Download Guide'. Again doesn't end with .imp.

Example #1 (the one you haven't nailed down yet) can rarely be found, so rare that I had trouble finding a single example to surf to, until I edited a message here and surfed to that link. So do we need to handle this situation? Maybe not!

Ashish, could you check the booklist creation code. Whenever there are more than 100 books IN TOTAL being sent (i.e. 101+ ), the booklist doesn't get processed. For 100 or below books it is fine!

My logs between the REB1200 and GEBLibrarian showed that the booklist should be sent in chunks of 100 books (each with its respective booklist length) at a time and then the INDEX=1 URL encoded tag will properly be bumped up to INDEX=101 for the next batch of 100 books and so on. Right now the INDEX gets bumped up by a very large amount and hence causes that aborted booklist attempt. This is an excerpt of my GEBLibrarain log that I was referring to:GET http://bookshelf.ebooksystem.net/bookshelf/default.asp?INDEX=1&REQUEST=100&SHOW_HIDDEN=YES HTTP/1.0
User-Agent: Allegro HTTP Client
Host: bookshelf.ebooksystem.net
Accept-Encoding: deflate, x-des-encrypted

...

GET http://bookshelf.ebooksystem.net/bookshelf/default.asp?INDEX=101&REQUEST=100&SHOW_HIDDEN=YES HTTP/1.0
User-Agent: Allegro HTTP Client
Host: bookshelf.ebooksystem.net
Accept: text/x-booklist
Accept-Encoding: deflate, x-des-encrypted

ashkulz

08-30-2008, 07:57 PM

NOTE: impserve has now been renamed to ebookutils, and is hosted at http://gitorious.org/projects/ebookutils using Git, which should easily allow other developers to make changes. I plan to merge impserve, impmake, 2 new python utilities which replace ebw1150+rebcomm and pdfread under this umbrella.

Changes in ebookutils-0.3:
- impserve: handle paging of booklist requests
- impserve: handle downloading of IMP books which are sent as an attachment (direct download of IMP from mobileread works)
- impserve: implement a plugin system and provide 2 default plugins to change the URL/content before being sent to the device.
- impmake: add an -f option to specify the files to be included, with one file per line
- impserve: refactored the code heavily, please run impserve/run.py from python.

ashkulz

08-30-2008, 08:09 PM

Ok, the only functionality I can think of which remains is:

[Ret] Handle internet links within ebooks (do the redirection which ebooksystem.net)
[nrapallo] add back button functionality
=> for this I plan to add a plugin point where you can hook into all requests coming for ebooksystem.net and refactor the code to use it. Plugins can then be written with that functionality in mind, without touching the core code.

[Ret] Let the reader delete ebooks from the online bookshelf.
why do we need that, really? just remove it from your filesystem, and next time you visit the booklist it's gone (it's autorefreshed on every request).

Any other feature requests?

ashkulz

08-31-2008, 12:54 PM

I've released ebookutils-0.4 having changes only in impserve. I consider it feature-complete for now, so pending any more feature requests I don't think it's going to change much.

Changes in ebookutils-0.4:
- impserve: redirect to the content page if bookstore/directory requested (remove the static page)
- impserve: allow books to be deleted from online bookshelf
- impserve: add support for back button with directory/store icon (going to online bookshelf erases history)
- impserve: make the proxy server single-threaded
- impserve: add support for browsing from links in ebooks, but it looks
broken like it triggers a bug in the firmware and gives a 400 error. Please try it at your end and let me know.

shighfield

08-31-2008, 07:10 PM

This is going to replace the wBookwise Libraian I think. :)

nekokami

09-01-2008, 12:13 AM

ashkulz

09-01-2008, 01:49 AM

I haven't tried it yet (my daughter has my ebw1150), but does it (or could it) pipe non-DRM versions of other format books through a converter to IMP so those could be downloaded through this system as well? I'm thinking of Gutenberg books, and possibly even CBZ. :D Yes, that can be done. All that would need to be done is write a plugin which intercepts the URL or format, do the conversion (via calling impmake on the command line) and return the converted book. That's one of the main reasons I built the plugin system :)

However, eBook Publisher is still very particular about the HTML it gets and crashes on invalid HTML (which is out there a lot) so that'd have to be run through HTML Tidy + doing misc cleanup on it.

Let me know what kind of conversion you envisage, and I can (attempt) to write the plugin or you can do it yourself, if you want ... ;)

ashkulz

09-01-2008, 01:57 AM

This is going to replace the wBookwise Libraian I think. :) Thanks, shighfield :) Looks like no one else is downloading/using it for now as the earlier versions work equally well..

Ret

09-01-2008, 09:11 AM

Thank you again ashkulz! Great effort! In such a short time you've made a great tool!:clap:

[Ret] Let the reader delete ebooks from the online bookshelf.
why do we need that, really? just remove it from your filesystem, and next time you visit the booklist it's gone (it's autorefreshed on every request).

I see you included this in version 0.4 but let me describe different scenarios where it can be useful: Even though I don't use that feature myself, sometimes you want to just delete a book while browsing your bookshelf, it's easier. Besides sometimes filenames are "cryptic", not including author or bookname, while on the reader all that information is displayed so it's easier to find the book.

Cheers

ashkulz

09-01-2008, 09:46 AM

Ret

09-01-2008, 04:52 PM

Yes, The same happens to my reader.
But there's something weird: While I also found the same crashes with privoxy/linreb solution, I could surf to the link to Google.CA inside nrapallo's test ebook and know it directly crashes the reader.

nrapallo

09-01-2008, 05:19 PM

BTW, can someone verify that browsing from ebooks works? the data is served from impserve, but it apparently triggers a bug in the ebook and it asks to reset itself ... can you verify that the same happens with you?

Oh, I can verify that they do work on the REB1200! I haven't yet tried this on an EBW1150, though.

I tried that next_1200.imp (http://www.mobileread.com/forums/attachment.php?attachmentid=15797&d=1220068009) I posted earlier and it worked flawlessly surfing to the wiki page.

If it doesn't work for you, what firmware is your REB1200? Mine is at 4.2f22.

If it would help, can you capture/generate a privoxy-like log of the HTTP packets sent/received? I can then match with mine, if there is something gone amiss.

The simulation of a 'back button' during surfing using the 'directory icon' works well, however, after the 'Online Bookshelf' tab is clicked and then the 'Bookstore icon' is clicked, nothing happens, but the last (html) page visited is displayed. Another 'directory icon' click brings up the main content page, but this should have happened when the 'Bookstore icon' was just clicked. Clear? :)

The text substitute of <a href> to include those UNDERLINE='YES' is a very useful feature. Any quick help/start guide you could provide on how to mod/write these plugins? I'm eager to try...

The .imp ebook transfer methods (#2 and #3 above) work just great. I was able to directly download a .imp book from the upload ebook forum here as well as a .imp from the the same directory Mobileread.com set up for the Mobipocket Download Guide. This is a great feature that ashkulz has "built-in' to ebookutils!!!! :2thumbsup

Major thanks for a job well done!!! :thumbsup:

Initial bug report:
- booklist gets truncated at 100 .imp books when the 'Online Bookshelf' (directory) has more than 99 ebooks. You just knew I was going to try this... (actually I had wanted to try with 10,000+ .imp books but settled for a directory with just 180 .imp ebooks in addition to my local shelf directory)

nrapallo

09-01-2008, 08:10 PM

I recently put my EBW1150 through a more exhaustive surf of the internet and found out it was not really up to the task of 'general' surfing; primarily due to it's more limited (available) internal memory.

The following is information I was given by ETI when I had asked about the EBW1150's (in)ability to display large .imp ebooks. Garth Conboy's response was: The ETI-1 (REB1200) has 8MB of RAM and it's all used for "data" -- the operating system reader application runs out of NOR flash.

The ETI-2 (EBW1150) is hugely memory constrained -- it has 4MB of RAM and that must be used for both code and data. So, when this device "exhausts an internal resource" it running out of RAM (heap space).

So the capacity of the ETI-2 to handle huge titles is certainly lower than that of the ETI-1.

I think we hit that "wall" way too soon when surfing the internet with the EBW1150 and it leads, all too often, to a "crash (http://www.mobileread.com/forums/showthread.php?p=241834#post241834)".

The ability to surf the internet is handled much better on the REB1200, primarily, I think, due to the internal memory differences noted above.

In conclusion, we must be careful how the EBW1150 is used to surf the internet and may have to limit ourselves to smaller, PDA-friendly pages.

Anyone else experience any different?

p.s. I'm afraid my EBW1150 also could not handle internet hyperlinks within a .imp ebook loaded into the reader; unlike the REB1200 which can.

ashkulz

09-02-2008, 08:39 AM

Oh, I can verify that they do work on the REB1200! I haven't yet tried this on an EBW1150, though.

I tried that next_1200.imp (http://www.mobileread.com/forums/attachment.php?attachmentid=15797&d=1220068009) I posted earlier and it worked flawlessly surfing to the wiki page.

If it doesn't work for you, what firmware is your REB1200? Mine is at 4.2f22.

If it would help, can you capture/generate a privoxy-like log of the HTTP packets sent/received? I can then match with mine, if there is something gone amiss. I don't remember my firmware version -- and I tried it with a different file I created. Let me do try both things and confirm that it works :-)

The simulation of a 'back button' during surfing using the 'directory icon' works well, however, after the 'Online Bookshelf' tab is clicked and then the 'Bookstore icon' is clicked, nothing happens, but the last (html) page visited is displayed. Another 'directory icon' click brings up the main content page, but this should have happened when the 'Bookstore icon' was just clicked. Clear? :) Can't do anything about it. The eBook caches that page and never requests from the server (see the server logs in the command prompt). When you request again, it hits impserve and then I do the correct thing. File a bug report with eBook Technologies to fix it ;-)

The text substitute of <a href> to include those UNDERLINE='YES' is a very useful feature. Any quick help/start guide you could provide on how to mod/write these plugins? I'm eager to try... See the provided plugins in the source code (http://gitorious.org/projects/ebookutils/repos/mainline/trees/master/impserve/plugins). e.g the one which does the underlines (and other things) is just 19 lines (http://gitorious.org/projects/ebookutils/repos/mainline/blobs/master/impserve%2Fplugins%2FAdjustHtml.py) -- all you have to do is derive from impserve.ProxyResponse and implement a get_response() method. You also might want to look at the documentation for BeautifulSoup (http://www.crummy.com/software/BeautifulSoup/documentation.html) -- it is a very powerful library...

Initial bug report:
- booklist gets truncated at 100 .imp books when the 'Online Bookshelf' (directory) has more than 99 ebooks. You just knew I was going to try this... (actually I had wanted to try with 10,000+ .imp books but settled for a directory with just 180 .imp ebooks in addition to my local shelf directory) I added code to implement that -- can you post the server log here so I can look into it as I don't have too many .IMP books right now :)

ashkulz

09-02-2008, 08:50 AM

I recently put my EBW1150 through a more exhaustive surf of the internet and found out it was not really up to the task of 'general' surfing; primarily due to it's more limited (available) internal memory.

The following is information I was given by ETI when I had asked about the EBW1150's (in)ability to display large .imp ebooks. Garth Conboy's response was:

I think we hit that "wall" way too soon when surfing the internet with the EBW1150 and it leads, all too often, to a "crash (http://www.mobileread.com/forums/showthread.php?p=241834#post241834)".

The ability to surf the internet is handled much better on the REB1200, primarily, I think, due to the internal memory differences noted above.

In conclusion, we must be careful how the EBW1150 is used to surf the internet and may have to limit ourselves to smaller, PDA-friendly pages.

Anyone else experience any different?

p.s. I'm afraid my EBW1150 also could not handle internet hyperlinks within a .imp ebook loaded into the reader; unlike the REB1200 which can. Hmm, what if we changed impserve so that pages above a certain limit are automatically paged (i.e. there is a limit on the HTML we send). Also, we can remove not understood HTML constructs (eg. script tags, frames) and compact it + run it through HTML tidy before being sent to the client? This can easily be done as a plugin (hint to nrapallo ;) )

nrapallo

09-02-2008, 09:17 AM

I added code to implement that -- can you post the server log here so I can look into it as I don't have too many .IMP books right now :)

Here's my log, but it looks a bit sparse.

Any debug code I could add, to get to the bottom of this behaviour?

ashkulz

09-02-2008, 09:21 AM

Here's my log, but it looks a bit sparse.

Any debug code I could add, to get to the bottom of this behaviour? Don't you get any "next" button or anything like that? because the request is for INDEX=1&REQUEST=100 which is what I serve ... can you post content of how GEB librarian sends the additional books.

nrapallo

09-02-2008, 09:24 AM

Don't you get any "next" button or anything like that? because the request is for INDEX=1&REQUEST=100 which is what I serve ... can you post content of how GEB librarian sends the additional books.

No, no next button.

For the GEBLibrarian log, I did above, see http://www.mobileread.com/forums/showthread.php?p=243832#post243832

I will try to get my original log if this is too "watered" down...

ashkulz

09-02-2008, 09:25 AM

do you get any next button if it sent by GEB Librarian?

Nate the great

09-02-2008, 09:27 AM

p.s. I'm afraid my EBW1150 also could not handle internet hyperlinks within a .imp ebook loaded into the reader; unlike the REB1200 which can.

My 1150 did it.

One ebook I downloaded from here had a link to www.mobileread.com. I clicked it and came here. I then directly downloaded several ebooks.

ashkulz

09-02-2008, 09:28 AM

My 1150 did it.

One ebook I downloaded from here had a link to www.mobileread.com. I clicked it and came here. I then directly downloaded several ebooks. We really need to see under which conditions it works and which it doesn't!

nrapallo

09-02-2008, 09:38 AM

do you get any next button if it sent by GEB Librarian?

No, GEBLibrarian didn't present me with a 'next' button either, but did download more than 100 ebooks.

Attached please find my 'raw' capture log with GEBLibrarian (of the edited one I presented above).

Nate the great

09-02-2008, 09:57 AM

We really need to see under which conditions it works and which it doesn't!

Here is what I know so far:

1, the ebooks were all made by JSWolf
2, all were created using Bookdesigner
3, before I clicked on a link, I ran impserve and looked at my "online bookshelf" (the one time I did not crashed my 1150)
4, the links all had http:// included

I do not know about the source format for the ebooks but I suspect it was HTML. I will go ask.

Nate the great

09-02-2008, 10:12 AM

5, Most of the ebooks were first made in Mobipocket format in BookDesigner. Then mobi2imp was used to convert to IMP.

nrapallo

09-02-2008, 10:17 AM

This can easily be done as a plugin (hint to nrapallo ;) )

:eek: Oh, nice, now I get volunteered by the Master... How can I say no... ;)

nrapallo

09-02-2008, 10:21 AM

Here is what I know so far:

1, the ebooks were all made by JSWolf
2, all were created using Bookdesigner
3, before I clicked on a link, I ran impserve and looked at my "online bookshelf" (the one time I did not crashed my 1150)
4, the links all had http:// included

I do not know about the source format for the ebooks but I suspect it was HTML. I will go ask.

5, Most of the ebooks were first made in Mobipocket format in BookDesigner. Then mobi2imp was used to convert to IMP.

Thanks Nate! Time to roll up those sleeves...

p.s. the next.imp and next_1200.imp (http://www.mobileread.com/forums/showthread.php?p=242537#post242537) where created with eBook Publisher from .html and all include the HTTP:// prefix. No Mobi2IMP was used.

nrapallo

09-02-2008, 11:09 AM

This can easily be done as a plugin (hint to nrapallo)

:eek: Oh, nice, now I get volunteered by the Master... How can I say no... ;)

I have had a lot of experience "cleaning up" html code to properly prepare a .imp ebook, but unfortunately, it is mostly from writing/using Mobi2IMP in perl.

To aid in the python plug-in cause, I offer simple text substitutes that could help ensure a html webpage/file will convert well to .imp (and hopefully better handled by our reader's internal browser).

Below please find a perl script fragment from Mobi2IMP.pl v9.4bwhere $html is the html (text) webpage
and $opt_NAME is an option you may wish to pass through
and $booktitle and $author could hold favortie hyperlinks in header or footers, if this works!

Just to get you started on those useful plug-ins...

################################################## #################

my $headerhr = "\n<HEADER><table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" width=\"100%\"><tr>\n";
$headerhr .= "<td align=\"left\" style=\"font-family:smallfont\"><small>" . $booktitle . "</small></td>\n";
$headerhr .= "<td align=\"right\" style=\"font-family:smallfont\"><small>" . $author . "</small></td></tr></table><hr></HEADER>\n";

my $headercolor = "\n<HEADER><table border=\"0\" cellpadding=\"0\" cellspacing=\"0\" width=\"100%\"><tr>\n";
$headercolor .= "<td align=\"left\" style=\"font-family:smallfont\" bgcolor=\"" . $opt_header_color . "\"><small>" . $booktitle . "</small></td>\n";
$headercolor .= "<td align=\"right\" style=\"font-family:smallfont\" bgcolor=\"" . $opt_header_color . "\"><small>" . $author . "</small></td></tr></table></HEADER>\n";

if (defined $opt_bgcolor) {
$html =~ s/<body([^>])*>/\n<BODY bgcolor=$opt_bgcolor>\n/i; #remove .mobi defaults in <body> and insert bgcolor
} else {
$html =~ s/<body([^>])*>/\n<BODY>\n/i; #remove .mobi defaults in <body>
}

if (defined $opt_header_hr) {
$html =~ s/<body([^>])*>/<BODY$1>\n$headerhr/i; #remove .mobi defaults in <body> and insert header-hr
} elsif (defined $opt_header_color) {
$html =~ s/<body([^>])*>/<BODY$1>\n$headercolor/i; #remove .mobi defaults in <body> and insert header-color
}

if (defined $opt_nopara and not defined $opt_noBRfix) {
$html =~ s/<br([^>])*><div/<BR \/><BR \/><div/gi; #force <br /> to work in Ebook Publisher
}

if (defined $opt_indent) {
#indent (~2 characters)
if (defined $opt_nopara) {
$html =~ s/<\/head>/<STYLE type="text\/css">p {text-indent:1em; margin-top:0em; margin-bottom:0em} header {display:none; display:oeb-page-head}<\/STYLE><\/head>/i; #nopara separation (--nopara)
} else {
$html =~ s/<\/head>/<STYLE type="text\/css">p {text-indent:1em; padding-top:0em; padding-bottom:1em} header {display:none; display:oeb-page-head}<\/STYLE><\/head>/i; #para separation (default)
}
} else {
#noindent (default)
if (defined $opt_nopara) {
$html =~ s/<\/head>/<STYLE type="text\/css">p {text-indent:0em; margin-top:0em; margin-bottom:0em} header {display:none; display:oeb-page-head}<\/STYLE><\/head>/i; #nopara separation (--nopara)
} else {
$html =~ s/<\/head>/<STYLE type="text\/css">p {text-indent:0em; padding-top:0em; padding-bottom:1em} header {display:none; display:oeb-page-head}<\/STYLE><\/head>/i; #para separation (default)
}
}

my $LRmargins = "2%";
if (defined $opt_LRmargins) { $LRmargins = $opt_LRmargins; }
if (defined $opt_nomargins) { $LRmargins = "0%"; }
if (defined $opt_nojustify) {
#nojustify body text (left-align)
if (defined $opt_smallerfont) {
$html =~ s/<body/<BODY style="margin-left:$LRmargins; margin-right:$LRmargins; font-size:x-small; text-align:left"/i; # add small margins and left-align text
} elsif (defined $opt_largerfont) {
$html =~ s/<body/<BODY style="margin-left:$LRmargins; margin-right:$LRmargins; font-size:medium; text-align:left"/i; # add small margins and left-align text
} else {
$html =~ s/<body/<BODY style="margin-left:$LRmargins; margin-right:$LRmargins; text-align:left"/i; # add small margins and left-align text
}
} else {
#justify body text (default)
if (defined $opt_smallerfont) {
$html =~ s/<body/<BODY style="margin-left:$LRmargins; margin-right:$LRmargins; font-size:x-small; text-align:justify"/i; # add small margins and justified text
} elsif (defined $opt_largerfont) {
$html =~ s/<body/<BODY style="margin-left:$LRmargins; margin-right:$LRmargins; font-size:large; text-align:justify"/i; # add small margins and justified text
} else {
$html =~ s/<body/<BODY style="margin-left:$LRmargins; margin-right:$LRmargins; text-align:justify"/i; # add small margins and justified text
}
}

$html =~ s/�//gi; # remove odd insertion of null chars
$html =~ s/<mbp:pagebreak/<p style="page-break-before:always"/gi; # insert proper page-breaks
$html =~ s/<mbpagebreak/<p style="page-break-before:always"/gi; # insert proper page-breaks
$html =~ s/<img align="baseline"/<img/gi; # remove the troublesome baseline keyword
$html =~ s/(<BR \/><BR \/>)+<div align="center"><img/<BR \/><div align="center"><p align="center"><img/gi; # only allow one <br> before an image to avoid after a page-break
$html =~ s/<div align="center"><img/<div align="center"><p align="center"><img/gi; # kludge to get eBook Publisher to center images

#fix up blank lines (unwanted) before page-break
$html =~ s/((<div([^>])*>( )*<\/div>)*(\s)*(<br([^>])*>)*(\s)*)*<p style="page-break-before/\n<p style="page-break-before/gi;
$html =~ s/((<br([^>])*>)*(\s)*)*<p style="page-break-before/\n<p style="page-break-before/gi;

$html =~ s/(<p style="page-break-before:always">)*<\/body>/<\/body>/gi; #fix up last (unwanted) page-break
$html =~ s/((<br([^>])*>)*(\s)*)*<\/body>/\n<\/body>/gi; #fix up blank lines (unwanted) at end

$html =~ s/<p/\n<p/gi; # insert newline before '<p' construct

################################################## #################

ashkulz

09-02-2008, 11:42 AM

nrapallo

09-02-2008, 11:52 AM

Mine is at 4.2f17. How do you upgrade your firmware? I tried "update now" but it gives a auto connect server timeout error.

My original REB1200 is 4.2f17 as well. I will try it to see if the firmware makes a difference. The REB1200 with the 4.2f22 firmware was a replacement I bought last year (just for the working battery) and was originally at firmware 2.0! I sent it into ETI and they sent it back with 4.2f22 (with no noticeable differences!). I too cannot get my original REB1200 to update to 4.2f22. I'll post my findings here, once I have a chance to do this.

Ok, I found the problem. It was a off-by-one bug in my code. You can see the change here (http://gitorious.org/projects/ebookutils/repos/mainline/commits/651adb81a10b3a1003a136fe988bf267d759d1b3) or can download the file to replace (impserve/impserve/__init__.py (http://gitorious.org/projects/ebookutils/repos/mainline/blobs/raw/651adb81a10b3a1003a136fe988bf267d759d1b3/impserve%2Fimpserve%2F__init__.py))

Good to know! Missed it by that much, I see. :rofl:

Have you downloaded 100+ ebooks from here yet, to try this out yourself? :grin2: :grin2: :grin2:

ashkulz

09-02-2008, 12:19 PM

BTW, I wish that someone would make this thread "sticky" (IMHO, it's more interesting than impmake which is sticky).

EDIT: well, that was fast. I asked and it became sticky in just a few minutes. Thanks, whoever it was :)

ashkulz

09-02-2008, 12:28 PM

Changes in ebookutils-0.4.1:
- impserve: fix off-by-one error which caused only 99 books to show in online bookshelf

nrapallo

09-02-2008, 01:36 PM

Changes in ebookutils-0.4.1:
- impserve: fix off-by-one error which caused only 99 books to show in online bookshelf

Yep, this worked for me. Now getting more than 100 ebooks from my Online Bookshelf. :thumbsup:

However, not been able to get my REB1200 with firmware 4.2f17 to use the next_1200.imp I created to surf the internet from within a .imp ebook. Also, these next.imp and next_1200.imp files caused strange linkage/reference problems within my Online Bookshelf. They were checked but not yet loaded onto the reader. Clicking each 'checkmark' icon caused only one to open. I could delete them though (and I did!).

Then I downloaded the next_1200.imp using the REB1200's surfing capabilities and had it expand the ebook 'on-the-fly'. Afterwards, I opened it on the reader and selected a hyperlinks, only to be met with a reset message 600 (it said "Your eBook needs to be reset. To do so, simply tap 'Continue'." Error (600))

I also got this reset from JSWolf's Stross, Charles: Accelerando (http://www.mobileread.com/forums/showthread.php?t=27686) when accessing the www.MobileRead.com link... This didn't work on the EBW1150 either!

Though, I DID NOT have any reset's with my other REB1200 with firmware 4.2f22!!! It works OK there! Could it be this is the answer? Both had approx. 20% free CF card space with the 4.2f22's CF card being 256 MB and the 4.2f17's CF Card being 1 GB. What else could cause this effect?

:chinscratch: Going to have to experiment some more! Anyone else want to try....

nrapallo

09-02-2008, 02:06 PM

Ashish,

I got the attached error when surfing to a MobileRead Forum and after loading the MR logo.gif. :chinscratch:

Could this be causing some de-stabilization within the reader... :chinscratch:

ashkulz

09-02-2008, 02:15 PM

Ashish,

I got the attached error when surfing to a MobileRead Forum and after loading the MR logo.gif. :chinscratch:

Could this be causing some de-stabilization within the reader... :chinscratch: That's very unlikely; the error you got is a normal socket error which you get when the client or server closes the link earlier than expected.

I think that the issue is related to the firmware; Garth Conboy at eBook Technologies has (kindly) promised to send me a link to upgrade to 4.2f22. I'll update y'all once I hopefully complete the upgrade :)

ashkulz

09-02-2008, 02:24 PM

nekokami

09-02-2008, 02:27 PM

Wow, nice to see that level of support from ETI, especially for an older product!

nrapallo

09-02-2008, 02:28 PM

I just now upgraded to 4.2f22 and browsing from the ebook works now :) So the solution for the 1200 is to upgrade to 4.2f22. For the 1150, I don't know what is the minimum firmware version required (or if it is OK for everyone).

There were a lot of fixes mentioned before I was able to upgrade, so I'd recommend everyone to upgrade anyway (Thanks to Garth at eBook Technologies, who arranged the upgrade within hours of me sending a mail).

Re: link to REB1200 4.2f22 firmware upgrade. Can you post it here? Never mind, Garth has already uploaded it for download here (http://www.ebooktechnologies.com/migration.htm)! Talk about a class act!

ashkulz

09-02-2008, 02:33 PM

Re: link to 4.2f22 firmware upgrade. Can you post it here? I don't think so. I got it when I visited "Menu > Bookstore > Directory > Customer Support" after getting a mail from them. Also, from what I recall all firmware upgrades are digitally signed for the 1150/1200 so I don't know if my image would help you :(

I'll follow up with him if the firmware can be posted openly, or if it is only available on user request.

ashkulz

09-02-2008, 02:35 PM

Re: link to REB1200 4.2f22 firmware upgrade. Can you post it here? Never mind, Garth has already uploaded it for download here (http://www.ebooktechnologies.com/migration.htm)! Talk about a class act! I don't think that's the version, as that is the link I used when migrating my 1200 from firmware 3.1 [although I don't know if it's changed since then].

ashkulz

09-02-2008, 02:41 PM

Wow, nice to see that level of support from ETI, especially for an older product! Very true. He also mentioned that they will accept bug reports for both firmware versions, which I had specifically asked [NOTE: I didn't ask about eBook Publisher].

nrapallo

09-02-2008, 02:52 PM

Very true. He also mentioned that they will accept bug reports for both firmware versions, which I had specifically asked [NOTE: I didn't ask about eBook Publisher].

Already been documented for him in the sticky thread
ETI eBook Publisher : bug reports / workarounds / update requests (http://www.mobileread.com/forums/showthread.php?t=22238); if he wants to look....

I checked the migration firmwares at the ETI website and they appear to be at version 4.2f22 for the REB1200 and 4.2f17 for the EBW1150. Hope this helps. I will be upgrading my original REB1200 soon!

:smack: Did he ask how well this 1999/2000 hardware product surfs the 'net? ask you for any tips? :rofl:

JSWolf

09-02-2008, 02:52 PM

5, Most of the ebooks were first made in Mobipocket format in BookDesigner. Then mobi2imp was used to convert to IMP.
Nate, would you mind testing the latest version of Scroogled in IMP to see if it really does work when you click the MobileRead link? I know for sure that one went HTML (original) > Book Designer > HTML (from Book Designer) > Mobipocket Creator (HTML from Book Designer) > IMP (Mobi2IMP)

thunderchild

09-02-2008, 02:59 PM

I have installed python and the utilities. But I have no idea how to get this to work. I have IMPServe working and have put 2 .imp books in the contents folder but how do I redirect the reader to look at this file? I have never used Python so its all really confusing!!! I have the colour of magic that I am desperate to read it.

Any help is appreciated.

ashkulz

09-02-2008, 03:03 PM

First, you'll have to go to settings, Ethernet Connection (1200) or Phone Connection (1150) and then tick the HTTP proxy box and enter the host as your IP address (1200) or 127.0.0.1 (1150). Then go to the online bookshelf, and you can access those IMPs. (You can find your IP on Windows by typing "cmd /k ipconfig" in Start -> Run).

If you want to do regular browsing, then you can create an index.html in the content folder with links to your favorite sites e.g.

<html><body><h1>Links</h1>
<p><a href="http://google.com/">Google</a></p>
<p><a href="http://www.mobileread.mobi/">Mobileread</a></p>
</body></html>

You can then use Menu -> Directory to go to this landing page, and from there on explore the net!

thunderchild

09-02-2008, 03:17 PM

Ashkulz

Thanks for the help but I have put those details in the reader and when I go to the bookshelf it still only shows what in the official Ebookwise bookshelf. I have Impserve runniing when trying to access the these .imps

Any suggestions?

Nate the great

09-02-2008, 03:19 PM

ashkulz

09-02-2008, 03:19 PM

Do you get any messages in the impserve window when you access the online bookshelf? If not, your configuration might be wrong (did you remember to tick the HTTP Proxy checkbox?).

thunderchild

09-02-2008, 03:25 PM

I have checked the HTTP proxy box and entered 9090. I get the message
"There was no response from the server;btrying the operation again may help"
It dose no tho.

Nate the great

09-02-2008, 03:27 PM

Nate, would you mind testing the latest version of Scroogled in IMP to see if it really does work when you click the MobileRead link? I know for sure that one went HTML (original) > Book Designer > HTML (from Book Designer) > Mobipocket Creator (HTML from Book Designer) > IMP (Mobi2IMP)

It worked, but the previous version worked better. In the version you uploaded this morning, the hyperlink has disappeared. I had to highlight the entire link before it would work. On the previous version, I simply clicked on it.

The current version has the current logo. The previous version has the "Saturn's Thumb" logo.

I found one way to make this not work. Whenever my 8MB storage was full, I kept getting data fault errors. Once I freed up about 3MB, I had much fewer errors.

ashkulz

09-02-2008, 03:29 PM

I have checked the HTTP proxy box and entered 9090. I get the message
"There was no response from the server;btrying the operation again may help"
It dose no tho. Okay, that's better ... I've assumed you've entered the host as 127.0.0.1. Can you tell me what message the impserve window gives (or at least post a screenshot of what it is showing)?

thunderchild

09-02-2008, 03:33 PM

Not sure how but it has now decided to work. I thank you for your help x

nrapallo

09-02-2008, 03:37 PM

Not sure how but it has now decided to work. I thank you for your help x

Hurray, another satisfied customer user. Let the testing begin...

ashkulz

09-02-2008, 03:37 PM

Not sure how but it has now decided to work. I thank you for your help x If you persevere, things are bound to work :thumbsup:

EDIT: this time Nick beat me to it!

Ret

09-02-2008, 04:06 PM

ashkulz

09-02-2008, 04:08 PM

What kind of an error do you get, Ret? I'd appreciate it if you could post the server logs...

Nate the great

09-02-2008, 04:16 PM

nrapallo

09-02-2008, 05:03 PM

I'm using EBW1150 with version 4.2f17 (updated it last week) and cannot surf the web from ebooks.
I rechecked using privoxy and I CAN surf to google Canana News link. I don't know if there's something going on in impserve's HTML parsing process. Maybe the file gets larger? I certainly don't know.

I have the same firmware version on my EBW1150 and also cannot surf the weblinks WITHIN a .imp ebook. It just resets.

However, surfing the 'net from the content/index.html works well; just that the EBW1150 is memory-constrained and crashes when it surfs 'too much', I think. EDIT: I want to confirm that this "crashing" is also exhibited when using the other 'surf the internet' method, Linreb+privoxy!

IF privoxy is enabled, can you THEN surf the 'net from the WITHIN a .imp ebook? What port configurations are you using for impserve, privoxy, and EBW1150 to make that happen! Was 'Google.ca News' the only link that worked? how about the wiki page?

:thumbsup: Ret, you always seems to have a solution that works!!!

ashkulz

09-02-2008, 05:12 PM

Hmm, you might want to contact ETI and ask if you can get 4.2f22 for the EBW-1150: the changelog that I saw included mention of fixes in the 1150 device also :)

ashkulz

09-02-2008, 11:16 PM

nrapallo

09-02-2008, 11:25 PM

Re: link to REB1200 4.2f22 firmware upgrade. Can you post it here? Never mind, Garth has already uploaded it for download here (http://www.ebooktechnologies.com/migration.htm)! Talk about a class act!

I'm using EBW1150 with version 4.2f17 (updated it last week) and cannot surf the web from ebooks.
I rechecked using privoxy and I CAN surf to google Canana News link. I don't know if there's something going on in impserve's HTML parsing process. Maybe the file gets larger? I certainly don't know.

OK, good news for REB1200 users. To be able to surf links from WITHIN a .imp ebook, upgrade your firmware to the latest (v4.2f22) at the ETI migration page here (http://www.ebooktechnologies.com/migration.htm). Just download the firmware .zip and expand the file into the root of an empty CF card (just to be safe). Follow the instructions and in less than five minutes you will be up and running again. If you had already registered your unit, then your User ID will be left intact and you WILL NOT not have to re-register your unit! All your exisiting ebooks are unaffected by this upgrade!

Now, some good news for EBW1150. We've seen that firmware 4.2f17 doesn't work yet with impserve and links WITHIN a .imp ebook. However, if you launch impserve (use port 9090) and then launch privoxy (listen to port 8118, by default), then the EBW1150 does surf the links within a .imp ebook. However, you must uncheck the 'Use HTTP proxy' setting on the EBW1150 just before you call that external link. It appears that privoxy takes over for impservve now and helps facilitate the redirect better than impserve does. I checked my next.imp ebook's link to the mobileread wiki page and it worked! Now to find out why!

I think this is what Ret was referring to above! While this is a kludge it does indicate that the EBW1150 can be made to works (in a limited way) in this situation despite it's firmware version.

Nate the great

09-02-2008, 11:36 PM

Except that my 1150 has 4.2f17, and I got links to work.

nrapallo

09-02-2008, 11:38 PM

Great news from Garth at ETI: We have opened up f22 for all users now.
So, the upgrade should be offered, or you could use the instructions below.

Best,
Garth So please try the Menu > Bookstore > Directory > Customer Support option and let us know if the upgrade works!

Something seems missing here for the EBW1150 with firmware 4.2f17. All I get at the end is the Customer Support page (like this (http://directory.ebooksystem.net/directory/default.asp)), with no option to download the latest firmware, anywhere. :chinscratch:

ashkulz

09-02-2008, 11:40 PM

Hmm, I'd appreciate it if Ret and nrapallo could post logs of both impserve and Privoxy so that I can help to debug it... there must be something in your environment that is different from Nate's.

Something seems missing here for the EBW1150 with firmware 4.2f17. All I get at the end is the Customer Support page (like this (http://directory.ebooksystem.net/directory/default.asp)), with no option to download the latest firmware, anywhere. :chinscratch: I don't know if there is a 4.2f22 for the 1150, I'm just guessing -- they might be using seperate version numbers for it (especially if they pushed an update last week).

nrapallo

09-03-2008, 12:01 AM

Except that my 1150 has 4.2f17, and I got links to work.

Does the above bold part of your message refer to links WITHIN a .imp ebook (and reproducible multiple times) as well as links from the content/index.html webpages.

Which ebook(s) did you try. I tried the next.imp and got the wiki page, Mobileread mobi page, GEBLibrarian home page working ONLY once I deactivated the 'Use HTTP proxy' setting, but left the 'Use proxy for USB' setting checked (and was using privoxy). Weird!

I run privoxy FIRST, then impserve with my local IP address and use port 9090. Privoxy listens on port 8118 and forwards *.ebooksystem.net to port 9090.

My same set-up (WinXP SP2) works with the REB1200 and by using privoxy with the EBW1150. I captured some logs using impserve with privoxy, but couldn't distinguish why it worked. I noted where I got a reader crash, specifically when I didn't use privoxy (as a safety net...).

Hope this helps narrow it down.

EDIT (09-03-2008): new logs to correct fact that privoxy is run before impserve to get this to work! Now 'third try' log is with NO privoxy software installed on my system and with impserve using host 127.0.0.1; just like Nate the great used! I can now surf links WITHIN a .imp ebook, but not for long before I see that 'crash'. :chinscratch:

Nate the great

09-03-2008, 12:21 AM

ashkulz

09-03-2008, 12:22 AM

do you have a proxy configured in internet explorer to use privoxy? impserve picks up your proxy settings from IE (which is automatically done by urllib, which I use).

ashkulz

09-03-2008, 12:23 AM

ashkulz

09-03-2008, 12:38 AM

nrapallo

09-03-2008, 12:40 AM

I've been navigating to MobileRead from a link inside the short story _Scroogled_. It doesn't work all the time, only about 75% (and the previous version works better than the most recent).

The settings I use are the ones I describe over in the front page thread. All I use is impserve (and python, of course). I don't use privoxy.

Also, I have used next.imp. The links in it worked.

Nate:

I tried mimicing your set-up and it worked great. :thumbsup:

I uninstalled the privoxy software from my system and rebooted. I also changed from using my local IP address for the EBW1150. Now I run impserve using host 127.0.0.1 (and port 9090). I can now surf links WITHIN a .imp ebook, but not for long before I see that 'crash'. Some webpages are probably too much for the EBW1150; that's OK, we'll stay away from them... :p

nrapallo

09-03-2008, 12:43 AM

Also, I'd like to make a suggestion: let's move the internet access related issues to the thread created by Nate (http://www.mobileread.com/forums/showthread.php?t=28623), so the solution/problem is known to everyone who may not read this thread to the very end :) [Plus that mentions the 1150 only, which is what we're having trouble with -- the 1200 seems to work fine once you upgrade to 4.2f22]

I agree, but after we nail this down between us pioneers... I'll post a quick how-to if we can get this working as Nate and I have just done (see previous message from both of us!).

Time to move it over already... :grin2:

ashkulz

09-03-2008, 12:45 AM

Nate:

I tried mimicing your set-up and it worked great. :thumbsup:

I uninstalled the privoxy software from my system and rebooted. I also changed from using my local IP address for the EBW1150. Now I run impserve using host 127.0.0.1 (and port 9090). I can now surf links WITHIN a .imp ebook, but not for long before I see that 'crash'. Some webpages are probably too much for the EBW1150; that's OK, we'll stay away from them... :p Great :thumbsup: Can you tell me what kind of pages you were loading? I'd like to isolate the cause so that we can either a) fix up the HTML being served, b) send the HTML in chunks, c) report a bug with the 1150 firmware (they said they'll accept bug reports) -- for the last we'll have to isolate a good test case so that they can reproduce it consistently.

ashkulz

09-03-2008, 12:50 AM

nrapallo

09-03-2008, 12:53 AM

Great :thumbsup: Can you tell me what kind of pages you were loading? I'd like to isolate the cause so that we can either a) fix up the HTML being served, b) send the HTML in chunks, c) report a bug with the 1150 firmware (they said they'll accept bug reports) -- for the last we'll have to isolate a good test case so that they can reproduce it consistently.

I just started with the next.imp ebook for the EBW1150 and choose the third link (test1), fourth (test2), fifth (test3 - sometimes unresponsive), test10 (sometimes resets)...

I think it has to do with local/internal ram being used up and the EBW1150 crashing when a subsequent (unpredictable) page gets served up [which uses up all available ram]. Just pure speculation here; no hard facts!

nrapallo

09-03-2008, 12:58 AM

BTW, a heads up: I'm in talks with Kovid to integrate ebookutils as plugins into Calibre (http://www.mobileread.com/forums/showthread.php?t=28573) (A patch to read .IMP and .RB metadata + import into library has already been merged). We still have to work out the details and implement it, so it may take a bit of time (but should get done, eventually).

You and Kovid discussed this over a year ago (for PDFRead) and finally we seem to be getting there (for ebookutils)! Good news!

No pressure here, but what about PDFRead being assimulated...

nrapallo

09-03-2008, 07:16 AM

I wrote (more like adapted) a plugin for impserve to force webpages to display in the smallest font, where possible.

This could be better tailored to make specific webpages work properly by adjusting some 'class' within a 'div'. Oh, well this is my first go with this idea. In essence, this is a proof-of-concept of a sorts that users can contribute and extend impserve's effectiveness!

Instructions:
=========
1. download the 'AdjustSmallestFont.py.txt' and rename 'AdjustSmallestFont.py'.
2. place the renamed file in your plugins directory (under your ebookutils/impserve directories).
3. run impserve as usual (I issue "run.py -a 127.0.0.1 -p 9090 shelf" from within a .bat file that I double-click. Yes, I'm lazy!)

Enjoy (and contribute back )! :thumbsup:

p.s. To modify AdjustSmallestFont.py to increase the xx-small font, change/edit "<font size='1'>" to "<font size='2'>" for x-small font; 3 for small; 4 for medium; 5 for large; and 6 for x-large.

ashkulz

09-03-2008, 07:56 AM

I just started with the next.imp ebook for the EBW1150 and choose the third link (test1), fourth (test2), fifth (test3 - sometimes unresponsive), test10 (sometimes resets)...

I think it has to do with local/internal ram being used up and the EBW1150 crashing when a subsequent (unpredictable) page gets served up [which uses up all available ram]. Just pure speculation here; no hard facts! Well, I'd appreciate it if someone could quantify it i.e. this site causes problems, or after loading HTML pages of XXX size the ebookwise resets or crashes, etc. Else we will not be able to use the internet browsing feature for the 1150 productively :)

You and Kovid discussed this over a year ago (for PDFRead) and finally we seem to be getting there (for ebookutils)! Good news!

No pressure here, but what about PDFRead being assimulated... Nope, at that point I was discussing integrating the 1100-related features, and this time it covers that and ebookutils. PDFRead was mentioned, but in a kind of "wouldn't it be great..." speculation. Assuming we get the design right, all kinds of conversion tools can be plugged into the pipeline (including PDFRead).

Please note that Calibre is NOT assimulating ebookutils or PDFRead: all we're discussing is a framework for providing plugins to do conversion tasks. Calibre will include some sample tool profiles for ebookutils, rbmake, etc. which can get activated if they're present and call the tools for doing the conversion tasks.

I wrote (more like adapted) a plugin for impserve to force webpages to display in the smallest font, where possible.

This could be better tailored to make specific webpages work properly by adjusting some 'class' within a 'div'. Oh, well this is my first go with this idea. In essence, this is a proof-of-concept of a sorts that users can contribute and extend impserve's effectiveness! Great to see you joining the club, and contributing with a plugin! I hope that you can continue with the plugin you mentioned that you were planning to develop ;)

run impserve as usual (I issue "run.py -a 127.0.0.1 -p 9090 shelf" from within a .bat file that I double-click. Yes, I'm lazy!)
You can directly double click on run.py and it will work, as long as the shelf directory is in the same directory tree -- all the command line switches are optional.

nrapallo

09-03-2008, 08:39 AM

Well, I'd appreciate it if someone could quantify it i.e. this site causes problems, or after loading HTML pages of XXX size the ebookwise resets or crashes, etc. Else we will not be able to use the internet browsing feature for the 1150 productively :)

It seems to not be that easy to quantify, I mean, if I start with test10 above first its OK, and if I then surf backwards, it will reset by the time I surf back to test2 or test1. The impserve logs don't show anything is wrong per se; the reader just resets. As I said before, this also did happen when I used the Linreb+privoxy method to surf. Perhaps I will try just the PDA-friendly links exclusively ( like www.mobileread.mobi ), and see how far I get. :blink:

I don't think this will affect the 'IMP Download Guide', that can now become a reality, thanks to the .imp ebook download 'on-the-fly' ability you coded into impserve.

Nope, at that point I was discussing integrating the 1100-related features, and this time it covers that and ebookutils. PDFRead was mentioned, but in a kind of "wouldn't it be great..." speculation. Assuming we get the design right, all kinds of conversion tools can be plugged into the pipeline (including PDFRead).

I had a vague recollection of "pipelines and such"; thanks for clearing this up for me.

Please note that Calibre is NOT assimulating ebookutils or PDFRead: all we're discussing is a framework for providing plugins to do conversion tasks. Calibre will include some sample tool profiles for ebookutils, rbmake, etc. which can get activated if they're present and call the tools for doing the conversion tasks.

Good to see that no "identity theft" will be involved here... :rofl:

Great to see you joining the club, and contributing with a plugin! I hope that you can continue with the plugin you mentioned that you were planning to develop ;)

Sorry, which plugin did you volunteer me to write, again? Oh, got to read back a few pages now... :grin2:

You can directly double click on run.py and it will work, as long as the shelf directory is in the same directory tree -- all the command line switches are optional.

You are even lazier than I ! :2thumbsup

ashkulz

09-03-2008, 08:49 AM

It seems to not be that easy to quantify, I mean, if I start with test10 above first its OK, and if I then surf backwards, it will reset by the time I surf back to test2 or test1. The impserve logs don't show anything is wrong per se; the reader just resets. As I said before, this also did happen when I used the Linreb+privoxy method to surf. Perhaps I will try just the PDA-friendly links exclusively ( like www.mobileread.mobi ), and see how far I get. :blink: I'll do some testing of my own sometime this weekend (I'll have to the borrow the 1150 which actually belongs to a relative). I don't really like to leave loose ends dangling ;)

Sorry, which plugin did you volunteer me to write, again? Oh, got to read back a few pages now... :grin2: I didn't volunteer you, you came up with the Gutenberg to IMP idea yourself!

nrapallo

09-03-2008, 09:07 AM

I'll do some testing of my own sometime this weekend (I'll have to the borrow the 1150 which actually belongs to a relative). I don't really like to leave loose ends dangling ;)

Using my EBW1150, I managed to surf the www.mobileread.mobi site quite well (50+ 'next' and then go to 'last' page and 10+ 'prev') with no resets in sight! I had my plugin to force the smallest font enabled and it crammed a lot of text onto the screen. I love it!

However, I was getting tired of plain-jane screens so I went to the forums and wiki and lo and behold I managed to get a crash! But I was looking for it to happen... See my log attached below, if it helps!

I didn't volunteer you, you came up with the Gutenberg to IMP idea yourself!

Oh that one! I thought you were referring to this one ( http://www.mobileread.com/forums/showthread.php?p=245206#post245206) and my (foolish) acceptance here (http://www.mobileread.com/forums/showthread.php?p=245263#post245263)... :rofl:

p.s. why does my log say "impmake 0.4.1: starting server on 127.0.0.1:9090" when running impserve... :chinscratch:

Ret

09-03-2008, 10:10 AM

I'll post the logs I get while trying to surf from nrapallo's test ebook. I try to follow link #9:
impserve:
Loaded plugin: AdjustHtml.py
Loaded plugin: AdjustQueryString.py
impmake 0.4: starting server on 0.0.0.0:9090
127.0.0.1 - - [03/Sep/2008 10:59:06] "GET http://bookshelf.ebooksystem.net/form/redirect.asp?target=http%3A%2F%2F72%2E14%2E205%2E1 04%2F&pubid=None%3Aebook%3Aguid-237ceaa3f5cb49b281ca33724a38bc5b HTTP/1.0" 302 -
127.0.0.1 - - [03/Sep/2008 10:59:06] "GET http://bookshelf.ebooksystem.net/content/ HTTP/1.0" 200 -
And EBW1150 crashes with error: "Data abort error at SPSR 200000 13 CPSR 20000097....."

While following link #7 (The one pointing to google canada news) I get:
Loaded plugin: AdjustHtml.py
Loaded plugin: AdjustQueryString.py
impmake 0.4: starting server on 0.0.0.0:9090
127.0.0.1 - - [03/Sep/2008 11:04:21] "GET http://bookshelf.ebooksystem.net/form/redirect.asp?target=http%3A%2F%2Fnews%2Egoogle%2Ec a%2Fnews%3Fned%3Dtca%26rec%3D0
&pubid=None%3Aebook%3Aguid-237ceaa3f5cb49b281ca33724a38bc5b HTTP/1.0" 302 -
127.0.0.1 - - [03/Sep/2008 11:04:21] "GET http://bookshelf.ebooksystem.net/content/ HTTP/1.0" 200 -
And the crash logs: "Prefetch abort fault at SPSR 200000 13 CPSR 20000097....."
This last link works "correctly" using privoxy.

Does this help?

Nate the great

09-03-2008, 10:15 AM

[QUOT=ashkulz;245704]Can you tell me which version worked better (linreb, or an older impserve version)? Also, can you post a log of where it fails so I can improve it :)[/QUOTE]

I was referring to the versions of the ebook. I will try to get to the log tonight.

Well, I'd appreciate it if someone could quantify it i.e. this site causes problems, or after loading HTML pages of XXX size the ebookwise resets or crashes, etc. Else we will not be able to use the internet browsing feature for the 1150 productively :)

I have been testing the link to MobileRead. It doesn't alway work, and I have come up with a few ideas.

a, If the 8MB storage is full or the link is inside a large IMP file, my 1150 will almost certainly crash.

b, If the link is inside a small file, my 1150 will likely not crash.

There could be other causes, but the two situations above have happened consistently.

nrapallo

09-03-2008, 10:22 AM

nrapallo

09-03-2008, 10:48 AM

a, If the 8MB storage is full or the link is inside a large IMP file, my 1150 will almost certainly crash.

You don't use a 128 MB SM card with the EBW1150? Why 8 MB?

b, If the link is inside a small file, my 1150 will likely not crash.

There could be other causes, but the two situations above have happened consistently.

I've tried the link to Mobileread.com in JSWolf's recent .imp ebooks and got it to work the one or two times I tried. I don't remember how big it was but I have a 128 MB SM card installed with almost 20% free.

Nate the great

09-03-2008, 11:01 AM

You don't use a 128 MB SM card with the EBW1150? Why 8 MB?

Crashing the 1150 seemed to screw up the card. It kept insisting it was half full when no books were on it.

You might want to clear out the 8MB storage on your device, and see if that helps.

nrapallo

09-03-2008, 11:12 AM

Crashing the 1150 seemed to screw up the card. It kept insisting it was half full when no books were on it.

You might want to clear out the 8MB storage on your device, and see if that helps.

Do you know how to format (erase external) the SM card within the ebook reader? You can format it using FAT (not FAT32) with a SM card reader and a PC, but I would still allow the reader to format it (again) before use, just to be sure.

The 8 MB internal storage is there in case there is no SM installed and it would be wise to keep it free of unnecessary files. When the SM card is installed, I think it overrides/supercedes its use.

ashkulz

09-03-2008, 11:20 AM

Using my EBW1150, I managed to surf the www.mobileread.mobi site quite well (50+ 'next' and then go to 'last' page and 10+ 'prev') with no resets in sight! I had my plugin to force the smallest font enabled and it crammed a lot of text onto the screen. I love it!

However, I was getting tired of plain-jane screens so I went to the forums and wiki and lo and behold I managed to get a crash! But I was looking for it to happen... See my log attached below, if it helps! That helps a lot, as it looks the ebook crashed just after trying to access the eBookwise photo (http://wiki.mobileread.com/images/thumb/b/bf/Eb1150large.gif/320px-Eb1150large.gif). Assuming that it needs to decode the whole picture in memory, then it requires roughly 0.5M of memory (a back of the envelope estimate). Can you tell me how much internal memory and how much free memory on the SM card was present? Because then it begins to look like the low memory issue is the root cause which may be turning up when a site has a lot of images, not when browsing mobile-friendly sites which anyway do not use images that much.

One quick test: can you create a new HTML in your content and include the image and see if it crashes if you navigate locally? If not, then does adding 2 or 3 different large images causes it to crash? If it does crash, then we need to benchmark the maximum size of the images allowed and if they are encountered, then do some handling in impserve for that.

p.s. why does my log say "impmake 0.4.1: starting server on 127.0.0.1:9090" when running impserve... :chinscratch: That's a typo which has already been fixed (http://gitorious.org/projects/ebookutils/repos/mainline/commits/943b692b6dff81c9f9c0c0c8e6ab4dadc6ae8bbf) yesterday but not put in the archive uploaded here.

Can you tell me which version worked better (linreb, or an older impserve version)? Also, can you post a log of where it fails so I can improve it :) I was referring to the versions of the ebook. I will try to get to the log tonight. Hmm, I don't think there's a way in which you can rollback to an older version of the firmware. If we could verify that, we can then report a regression to ETI...

I have been testing the link to MobileRead. It doesn't alway work, and I have come up with a few ideas.

a, If the 8MB storage is full or the link is inside a large IMP file, my 1150 will almost certainly crash.

b, If the link is inside a small file, my 1150 will likely not crash.

There could be other causes, but the two situations above have happened consistently. That ties in well with the analysis for nrapallo above. Can you try to remove all books from the internal storage and move them to the SM card? I suspect that the 1150 may be using the internal storage as swap space if it runs out of memory (which is more likely to happen in a large IMP file).

Do you know how to format (erase external) the SM card within the ebook reader? You can format it using FAT (not FAT32) with a SM card reader and a PC, but I would still allow the reader to format it (again) before use, just to be sure.

The 8 MB internal storage is there in case there is no SM installed and it would be wise to keep it free of unnecessary files. When the SM card is installed, I think it overrides/supercedes its use. I'd advise you to do the same, as we need to verify the swap space theory too...

Nate the great

09-03-2008, 11:52 AM

Hmm, I don't think there's a way in which you can rollback to an older version of the firmware. If we could verify that, we can then report a regression to ETI...

I was talking about the ebook file, not the ebook _reader_.

ashkulz

09-03-2008, 12:12 PM

I was talking about the ebook file, not the ebook _reader_. One more victim of dual-use words :blink:

nrapallo

09-03-2008, 12:35 PM

That helps a lot, as it looks the ebook crashed just after trying to access the eBookwise photo (http://wiki.mobileread.com/images/thumb/b/bf/Eb1150large.gif/320px-Eb1150large.gif). Assuming that it needs to decode the whole picture in memory, then it requires roughly 0.5M of memory (a back of the envelope estimate). Can you tell me how much internal memory and how much free memory on the SM card was present? Because then it begins to look like the low memory issue is the root cause which may be turning up when a site has a lot of images, not when browsing mobile-friendly sites which anyway do not use images that much.

It doesn't use the 8 MB internal storage (or external SM) as internal memory or swap area. The internal memory on the EBW1150 is 4 MB for both code (builtin reader app) and data per Garth's email to me and info posted here (http://www.mobileread.com/forums/showthread.php?p=245019#post245019). It's running out of internal (4 MB) memory when it loads that .gif image at the end of surfing; but not when I surf there first since I passed that point before (see my EBW1150 album)! I believe, the 4 MB internal memory is the culprit!

One quick test: can you create a new HTML in your content and include the image and see if it crashes if you navigate locally? If not, then does adding 2 or 3 different large images causes it to crash? If it does crash, then we need to benchmark the maximum size of the images allowed and if they are encountered, then do some handling in impserve for that.

I don't need to do that because the content directory I uploaded with Linreb-0.2 had a large picture (REBTD-fullpage.png @ 0.48 MB) followed by a smaller sized file of the same image (REBTD-fullpage256.png @ 0.13 MB) on the second page of the next.html which would crash the EBW1150 sometimes. The follower image (REBTD-fullpage256.png) would render without problems. So it now seems that I have seen this behaviour before... :cool:

A solution (for large images crashing the reader) could be for a plugin to 'convert' all images (over a certain size) to 16 or 256 colors, if possible. :nudge: :nudge: :wink: :wink:
That's a typo which has already been fixed (http://gitorious.org/projects/ebookutils/repos/mainline/commits/943b692b6dff81c9f9c0c0c8e6ab4dadc6ae8bbf) yesterday but not put in the archive uploaded here.

Hmm, I don't think there's a way in which you can rollback to an older version of the firmware. If we could verify that, we can then report a regression to ETI...

That ties in well with the analysis for nrapallo above. Can you try to remove all books from the internal storage and move them to the SM card? I suspect that the 1150 may be using the internal storage as swap space if it runs out of memory (which is more likely to happen in a large IMP file).

I'd advise you to do the same, as we need to verify the swap space theory too...

The (limited) 4 MB internal memory seems to be the culprit with reader resets. :)

ashkulz

09-03-2008, 01:39 PM

A solution (for large images crashing the reader) could be for a plugin to 'convert' all images (over a certain size) to 16 or 256 colors, if possible. :nudge: :nudge: :wink: :wink:

The (limited) 4 MB internal memory seems to be the culprit with reader resets. :) Do you think that this would help? Because in the end, I think that everything has to be rendered as a grayscale bitmap image (the screen display needs it) then it doesn't matter how many colors the input has, the output memory requirements may be the killing [pun intended] factor.

Some more things to try out: use imagemagick or Irfanview or <tool-of-choice> to actually do the conversion and see if it makes a difference. Also, does the format matter? i.e. GIF vs PNG vs JPG -- if you can convert all images to one format and see if using one results in less crashes i.e. less memory use, then it'd make more sense to develop a plugin that would resize the image to the 1150's display size and save in the least-memory-consuming format. I really wish I had that 1150 with me to test all this...

nrapallo

09-03-2008, 02:49 PM

I compiled the Mobileread Full List (http://www.mobileread.com/forums/ebooks.php?do=getlist&type=html) (with only .IMP ebooks) here at mobileread.com as of today (Sept 3, 2008) into ebook format.

Try and see if you can download it to your reader from here, If not, just download it to your computer and then upload it to your reader from there! then open the ebook on the reader and select a title therein.

This will surf to the upload thread with description and link to the actual .imp. Select the .imp link and it will be downloaded directly into your reader.

Now the fun begins, you have to find it under some category. Usually the 'General Interest' category is used so look there first.

This works great for me, as long as I strike out that part above, when using my EBW1150!

Have fun and give us some feedback on how well this works for you (state your reader and firmware if you experience problems)!

As usual, the ebook ending with _1200.imp is for the REB1200; the other .imp for the EBW1150.

nrapallo

09-03-2008, 09:02 PM

This ebook (http://www.mobileread.com/forums/showthread.php?t=28686) has been uploaded to the IMP ebook section to faciliate 'surfing the internet' using PDA Friendly links.

BTW, the best way to use the "back" button feature is to hit the 'last book' icon at the bottom left of your screen (not the directory icon!)

How timely! Just ignore any irrelevant links; and maybe "roll-your-own" and share...

ashkulz

09-04-2008, 04:18 AM

Nate the great

09-04-2008, 08:36 AM

ashkulz

09-04-2008, 10:37 AM

I have a suggestion. At some point are you going to add file conversion to impserve? Given that it is running on a computer, we do have the processing cycles to spare.

My suggestion is that you prepare for the new feature by having impserve download incompatible files (like ZIP and PRC) to the content folder. I'm open to suggestions on how this feature should work, and what exactly is it that is wanted. To be honest, I'd prefer that most of features like this should be done as plugins, which is why I added a plugin framework for impserve. If how to do such a plugin is not clear, I'm attaching a plugin which uses a 'hypothetical' command prc2imp to do the job.

# WARNING: this is untested code that I cooked up in 5 minutes. Take it only as an example!
import impserve
import os, subprocess

class PrcConverter(impserve.ProxyResponse):
def get_response(self, url, headers, data):
if not 'application/x-palm-prc' in headers['Content-Type']:
return headers, data

open('input.prc', 'w').write(data)

# assuming a command prc2imp exists
if not subprocess.call(['prc2imp', 'input.prc', '-o', 'output.imp']) or not os.path.isfile('output.imp'):
return headers, data

headers['Content-Type'] = 'application/x-softbook'
data = open('output.imp').read()
os.remove('input.prc')
os.remove('output.imp')
return headers, data

The only important parts are the 'Content-Type' part, which is what the server is returning as Content-Type. If you expect the server to get it wrong, add a file called mime.types in the impserve directory and add the following to it: application/x-palm-prc prc pdb

The other important part is where you do your conversion: here it is done via an external process (as mentioned earlier). Obviously, you'd have more code here (metadata extraction, error handling, etc) but requests like these should be handled as plugins, as some people may want them and some don't.

nrapallo

09-04-2008, 11:12 AM

... uses a 'hypothetical' command prc2imp to do the job.

Thanks for this headstart! A plugin it be, then. :thumbsup:

Oh, now you KNOW that this command will be 'Mobi2IMP (dos version)'; the GUI version would just interfer too much with the flow.

OK, just to remember where my Beginner's Guide to Python is... oh, right here on my REB1200... :p :)

nrapallo

09-04-2008, 02:06 PM

I've released version 0.4.2. Changes:
- correct a small typo when impserve reported itself as impmake while starting up
- changes for creating a standalone windows version via py2exe

Please note that you will require to install eBook Publisher seperately for using impmake, which hasn't been bundled. Thanks to nrapallo for the suggestion to create a windows-only version, which would be more helpful for people to get it up and running.

Another suggestion would be to include a 'standard' content startup page (index.html).

Oh, let me see, I should have one lying around here somewhere... oh, here it is! :cool:

It's the Impserve startup page (MobileRead edition)! Just extract index.html in the below content.zip and place it in your 'content' directory.

While you are at it, use the included impserve-start.bat to startup the program. It belongs in the same directory where impserve.exe resides! This assumes you are using the windows (win32) executables!

Also, useful posts to review once you start using impserve:AdjustSmallestFont.py (http://www.mobileread.com/forums/showthread.php?p=245787#post245787) First user-supplied plugin (forces smallest font) for impserve...

mrbooks-09-03-08.imp (http://www.mobileread.com/forums/showthread.php?p=246134#post246134) test .imp ebook for impserve direct .imp downloading...

Cousins, Adrienne: Kindle Bookmarks v0.11, Sep 3,2008 (http://www.mobileread.com/forums/showthread.php?t=28686) .imp ebook with PDA-friendly links...

Enjoy!

ashkulz

09-04-2008, 10:55 PM

Another suggestion would be to include a 'standard' content startup page (index.html).

Oh, let me see, I should have one lying around here somewhere... oh, here it is! :cool: I prefer not to provide a content page provided by default, as otherwise upgrades are very simple: just unzip over the existing installation and it works. Otherwise care would have to be taken by the user to NOT overwrite any files, which is not a good thing and if missed by someone would lead to their losing the changes they made :( However, I do like the idea of providing a seperate "content" package which provides this: it would be clear to most people that using it might overwrite any customizations they made.

EDIT: add a link to your post in the first post.

nrapallo

09-04-2008, 11:55 PM

Ashish:

I agree; choice to install content is better than no content...

Also, I tried your PrcConverter.py, but I got a message on startup that it failed to load. Any ideas why that would happen. I ran the script on its own to see if there were any syntax errors but only got the message that impserve wasn't defined.

I know you are away for a bit, but does anything appear wrong with this:# v0.1 - Inital plugin idea by ashkulz; Adapted by nrapallo (Sept 2008)
import impserve
import os, subprocess

class PrcConverter(impserve.ProxyResponse):
def get_response(self, url, headers, data):
if not 'prc' in headers['Content-Type']:
return headers, data

open('input.prc', 'wb').write(data)

# using a command to convert .prc to .imp 'on the fly'... copied "C:\Program Files\Mobi2IMP\bin\Mobi2IMP.exe" to impserve directory
if not subprocess.call(['Mobi2IMP', '--cover', '--o', 'output', 'input.prc']) or not os.path.isfile('output.imp'):
return headers, data

headers['Content-Type'] = 'application/x-softbook'
data = open('output.imp', 'rb').read()
os.remove('input.prc')
os.remove('output.imp')

return headers, data

I got this message when impserve was launched:Loaded plugin: AdjustHtml.py
Loaded plugin: AdjustQueryString.py
The following plugins failed to load: PrcConverter.py
impserve 0.4.2: starting server on 127.0.0.1:9090

ashkulz

09-05-2008, 12:04 AM

nrapallo

09-05-2008, 12:17 AM

Works for me: try downloading the attachment I've done here. ashish@hunger:~/sandbox/ebookutils/impserve$ python run.py
Loaded plugin: AdjustHtml.py
Loaded plugin: AdjustQueryString.py
Loaded plugin: PrcConverter.py
impserve 0.4.2-dev: starting server on 0.0.0.0:9090

It doesn't work with the 0.4.2 win32 executable, but does work with the python script. Now when I say work I mean it loads... :( The EBW1150 says it is an "Unknown document type" even though I have that 'mime.types' file with the 'application/x-palm-prc prc pdb' entry.

Back to debugging...

nrapallo

09-05-2008, 01:32 AM

It doesn't work with the 0.4.2 win32 executable, but does work with the python script. Now when I say work I mean it loads... :( The EBW1150 says it is an "Unknown document type" even though I have that 'mime.types' file with the 'application/x-palm-prc prc pdb' entry.

Back to debugging...

I now have a semi-functional PrcConverter.py but it does not load the resulting .imp onto the reader automatically yet!

My revised PrcConverter.py is:# v0.1 - Inital plugin idea by ashkulz; Adapted by nrapallo (Sept 2008)
import impserve
import os, subprocess

class PrcConverter(impserve.ProxyResponse):
def get_response(self, url, headers, data):
#debug statement (TODO: also check if .ext is .prc/.pdb/.mobi)
print headers
if not 'application/octet-stream' in headers['Content-Type']:
return headers, data
print "0\n" #debug statement

open('input.prc', 'wb').write(data)
print "1\n" #debug statement
# using a command to convert .prc to .imp 'on the fly'... copied "C:\Program Files\Mobi2IMP\bin\Mobi2IMP.exe" to impserve directory
if not subprocess.call(['Mobi2IMP', '--cover', '--out', 'output', 'input.prc']) or not os.path.isfile('output.imp'):
return headers, data
print "2\n" #debug statement
headers['Content-Type'] = 'application/x-softbook'
data = open('output.imp', 'rb').read()
os.remove('input.prc')
os.remove('output.imp')
print "3\n" #debug statement
return headers, data

Excuse the print / debug statements for now. They will be removed when this plugin works!

This is the plugin debug log produced:Loaded plugin: AdjustHtml.py
Loaded plugin: AdjustQueryString.py
Loaded plugin: PrcConverter.py
impserve 0.4.2: starting server on 127.0.0.1:9090
127.0.0.1 - - [05/Sep/2008 01:12:32] "GET http://authenticate.ebooksystem.net/authenticate/default.asp?SS=1&GUEST_ID=GC0000216225&DEVICE_TYPE=SoftBook&HWSERIES=2250&VERSION=35345&LANG=en&COLOR=NO&DIALED=UNKNOWN&GMT_OFFSET=-14280&SBPISP=NO&SE=1&MESSAGE=5757B0419BE4374C039A10FA38C4EDC70837BB0487 AA60F27C9420D03ED187CB&BOOKSHELF=YES HTTP/1.0" 302 -
127.0.0.1 - - [05/Sep/2008 01:12:32] "GET http://bookshelf.ebooksystem.net/content/ HTTP/1.0" 200 -
127.0.0.1 - - [05/Sep/2008 01:12:33] "GET http://www.google.com/intl/en/images/about_logo.gif HTTP/1.0" 200 -
Content-Type: image/gif
Last-Modified: Wed, 07 Jun 2006 19:37:54 GMT
Expires: Sun, 17 Jan 2038 19:14:07 GMT
Cache-Control: public
Date: Fri, 05 Sep 2008 05:09:44 GMT
Server: gws
Content-Length: 5548
Connection: Close

127.0.0.1 - - [05/Sep/2008 01:12:37] "GET http://www.mobileread.com/forums/forumdisplay.php?f=128 HTTP/1.0" 200 -
Connection: close
Set-Cookie: bblastvisit=1220591391; expires=Sat, 05-Sep-2009 05:09:51 GMT; path=/; domain=.mobileread.com
Set-Cookie: bblastactivity=0; expires=Sat, 05-Sep-2009 05:09:51 GMT; path=/; domain=.mobileread.com
Set-Cookie: bbforum_view=c35555c929d51e1050e745ebb1541863c3e44 6a6a-1-%7Bi-128_i-1220591391_%7D; path=/; domain=.mobileread.com
Expires: 0
Cache-Control: private, post-check=0, pre-check=0, max-age=0
Pragma: no-cache
Content-Type: text/html; charset=ISO-8859-1
X-UA-Compatible: IE=7
Date: Fri, 05 Sep 2008 05:09:51 GMT
Server: Apache

127.0.0.1 - - [05/Sep/2008 01:12:42] "GET http://www.mobileread.com/forums/images/mr/misc/logo.gif HTTP/1.0" 200 -
Connection: close
Content-Type: image/gif
Accept-Ranges: bytes
ETag: "216147554"
Last-Modified: Wed, 09 Apr 2008 09:29:42 GMT
Content-Length: 3364
Date: Fri, 05 Sep 2008 05:09:56 GMT
Server: Apache

127.0.0.1 - - [05/Sep/2008 01:12:42] "GET http://www.mobileread.com/forums/images/mr/misc/rss.gif HTTP/1.0" 200 -
Connection: close
Content-Type: image/gif
Accept-Ranges: bytes
ETag: "1463968026"
Last-Modified: Fri, 18 May 2007 09:42:24 GMT
Content-Length: 1024
Date: Fri, 05 Sep 2008 05:09:57 GMT
Server: Apache

127.0.0.1 - - [05/Sep/2008 01:12:46] "GET http://www.mobileread.com/forums/clear.gif HTTP/1.0" 200 -
Connection: close
Content-Type: image/gif
Accept-Ranges: bytes
ETag: "1146313854"
Last-Modified: Tue, 15 Jul 2008 14:20:24 GMT
Content-Length: 43
Date: Fri, 05 Sep 2008 05:10:00 GMT
Server: Apache

127.0.0.1 - - [05/Sep/2008 01:12:46] "GET http://www.mobileread.com/forums/images/mr/misc/f-129.png HTTP/1.0" 200 -
Connection: close
Content-Type: image/png
Accept-Ranges: bytes
ETag: "1469148012"
Last-Modified: Fri, 22 Jun 2007 14:29:15 GMT
Content-Length: 1458
Date: Fri, 05 Sep 2008 05:10:01 GMT
Server: Apache

127.0.0.1 - - [05/Sep/2008 01:12:54] "GET http://www.mobileread.com/forums/showthread.php?t=22663 HTTP/1.0" 200 -
Connection: close
Set-Cookie: bblastvisit=1220591408; expires=Sat, 05-Sep-2009 05:10:08 GMT; path=/; domain=.mobileread.com
Set-Cookie: bblastactivity=0; expires=Sat, 05-Sep-2009 05:10:08 GMT; path=/; domain=.mobileread.com
Expires: 0
Cache-Control: private, post-check=0, pre-check=0, max-age=0
Pragma: no-cache
Content-Type: text/html; charset=ISO-8859-1
X-UA-Compatible: IE=7
Date: Fri, 05 Sep 2008 05:10:08 GMT
Server: Apache

127.0.0.1 - - [05/Sep/2008 01:12:58] "GET http://www.mobileread.com/forums/image.php?u=8654&dateline=1186863588 HTTP/1.0" 200 -
Connection: close
Cache-control: max-age=31536000
Expires: Sat, 05 Sep 2009 05:10:12 GMT
Content-disposition: inline; filename=Sistine_Sibyl.jpg
Content-transfer-encoding: binary
Content-Length: 3723
Last-Modified: Sat, 11 Aug 2007 20:19:48 GMT
ETag: "1186863588-8654"
Date: Fri, 05 Sep 2008 05:10:12 GMT
Server: Apache
Content-Type: image/jpeg

127.0.0.1 - - [05/Sep/2008 01:12:59] "GET http://www.mobileread.com/forums/images/mr/attach/prc.gif HTTP/1.0" 200 -
Connection: close
Content-Type: image/gif
Accept-Ranges: bytes
ETag: "81527440"
Last-Modified: Fri, 22 Jun 2007 19:12:30 GMT
Content-Length: 135
Date: Fri, 05 Sep 2008 05:10:14 GMT
Server: Apache

127.0.0.1 - - [05/Sep/2008 01:13:02] "GET http://www.mobileread.com/forums/attachment.php?attachmentid=12111&d=1207787498 HTTP/1.0" 200 -
Connection: close
Vary: User-Agent
X-UA-Compatible: IE=7
Cache-control: max-age=31536000, private
Expires: Sat, 05 Sep 2009 05:10:16 GMT
Last-Modified: Thu, 10 Apr 2008 00:31:38 GMT
ETag: "12111"
Accept-Ranges: bytes
Content-disposition: attachment; filename="Pozzuoli_War%20of%20the%20Wenuses.prc"
Content-Length: 105285
Date: Fri, 05 Sep 2008 05:10:16 GMT
Server: Apache
Content-Type: application/octet-stream

0

1

Mobi2IMP (version 9.4b) Copyright (C) 2008 Nick Rapallo (nrapallo)
FULLFILENAME: input.prc
FILENAME : input.prc
EXPLODEPATH : .\
EXPLODEDIR : input
DatabaseName: Pozzuoli_War of the Wenuses
Version : 0
Type : TEXt
Creator : REAd
baktime : -2082844800
# of records: 20
---------------------------------------------------
---------------------------------------------------
Found 1 images
Looking for filepos
Found all 17 filepos
Adding name attributes
WARNING 17: 0000077137 - Not an anchor:
Fixing filepos attribute
EBW 1150 ebook created!

(reader responded "Unknown document type" at this point and the plugin failed to print "2")

(directory icon tapped to go back one webpage)

127.0.0.1 - - [05/Sep/2008 01:13:31] "GET http://directory.ebooksystem.net/directory/default.asp HTTP/1.0" 302 -
127.0.0.1 - - [05/Sep/2008 01:13:32] "GET http://www.mobileread.com/forums/forumdisplay.php?f=128 HTTP/1.0" 200 -
Connection: close
Set-Cookie: bblastvisit=1220591446; expires=Sat, 05-Sep-2009 05:10:46 GMT; path=/; domain=.mobileread.com
Set-Cookie: bblastactivity=0; expires=Sat, 05-Sep-2009 05:10:46 GMT; path=/; domain=.mobileread.com
Set-Cookie: bbforum_view=16ca0a403c69cd6c63057b53f7c6b162dd613 c38a-1-%7Bi-128_i-1220591446_%7D; path=/; domain=.mobileread.com
Expires: 0
Cache-Control: private, post-check=0, pre-check=0, max-age=0
Pragma: no-cache
Content-Type: text/html; charset=ISO-8859-1
X-UA-Compatible: IE=7
Date: Fri, 05 Sep 2008 05:10:46 GMT
Server: Apache

Note that I placed a copy of Mobi2IMP.exe from the bin directory created by the Mobi2IMP installer package. The plugin did manage to save the input.prc and create the output.imp, but didn't load it "on-the-fly". (See attached .prc & .imp)

Notice above that Mobileread returns a Content-Type of 'application/octet-stream' when requesting the final .prc file. It has the filename there too so the plugin should, in the future, check the .ext to make sure we are doing a valid conversion.

Also, it seems that Mobileread uses a mimetype of 'application/x-mobipocket-ebook' for .prc download requests per the more detailed log report created prior to the one above. (See below text file)

nrapallo

09-06-2008, 01:05 AM

I have a suggestion. At some point are you going to add file conversion to impserve? Given that it is running on a computer, we do have the processing cycles to spare.

My suggestion is that you prepare for the new feature by having impserve download incompatible files (like ZIP and PRC) to the content folder.

I now have a semi-functional PrcConverter.py but it does not load the resulting .imp onto the reader automatically yet!

:yahoo: I am a bit excited about this .prc to .imp plugin. It works seemlessly now. May be a bit crude as python coding goes, but it works flawlessly!

A great suggestion from Nate, turns out to be a great convenience to ALL ebookwise 1150 users (and the few REB1200 users here).

There are two versions of this plugin for impserve; EBW1150 users will want the 'PrcConverter1150.py'; REB1200 users the 'PrcConverter1200.py'. Don't intermix. You'll need a copy of Mobi2IMP.exe (dos command) to make this work; see here (http://www.mobileread.com/forums/showthread.php?t=22178) for Mobi2IMP-9.4b-dos.zip.

Instructions:
=========
1. For EBW1150 users, download the 'PrcConverter1150.py.txt' and rename 'PrcConverter1150.py'.
2. Place the renamed file in your plugins directory (under your ebookutils/impserve directories).
3. That's it! Download all the non-DRM'ed mobipocket/.prc/.pdb you want to read, onto your reader, using one-step.

Enjoy and pass this info onto a friend... Native .imp downloading and now .prc download as well! WOW, wow, wow.... :yahoo:

p.s. I also include BELOW updated versions of the AdjustSmallestFont.py (http://www.mobileread.com/forums/showthread.php?p=245787#post245787) for each reader (EBW1150 and REB1200).

ashkulz

09-06-2008, 08:29 AM

:yahoo: I am a bit excited about this .prc to .imp plugin. It works seemlessly now. May be a bit crude as python coding goes, but it works flawlessly!

A great suggestion from Nate, turns out to be a great convenience to ALL ebookwise 1150 users (and the few REB1200 users here).

[snip]

Enjoy and pass this info onto a friend... Native .imp downloading and now .prc download as well! WOW, wow, wow.... :yahoo:

p.s. I also include BELOW updated versions of the AdjustSmallestFont.py (http://www.mobileread.com/forums/showthread.php?p=245787#post245787) for each reader (EBW1150 and REB1200). This is fantastic! :goodjob: done by nrapallo ... looks like my evil plan to get more people involved via plugins is beginning to bear fruits :evilgrin: As of now, we have 3 (count them, 3) plugins written by nrapallo: AdjustHtmlSize, PrcConverter, and an unreleased AdjustImageSize which reduces the image size for saving low-memory situations on the 1150.

I've got some new infrastructure enhancements planned in impserve which I plan to implement after this weekend [till which I'm busy in real life]:
add support for ebook detection in impserve
serve only relevant [i.e. 1150/1200] IMPs in book list using ebook detection
allow the plugins to access which ebook is currently detected
refactor the plugin system so that a plugin can take over the whole request and redo the current "local" processing in terms of that
verify working of plugins in the win32 binary distribution
One more idea which I've had for a plugin is to insert a special link on all pages and on clicking it would go to a server page allowing one to create an IMP consisting of all the pages read till then, and name the book. The use-case would be simple: say you're reading a article on the net [or a series of articles], you "save" them one-by-one and when you're finished, click on generate and presto! you have a ready-made IMP which contains everything you just read till now for offline reading [admittedly, this one would be a much more complex plugin].

nekokami

09-06-2008, 02:29 PM

:yahoo: I am a bit excited about this .prc to .imp plugin. It works seemlessly now. May be a bit crude as python coding goes, but it works flawlessly!
Excllent. Now if someone could just get .cbz downloads working, my younger daughter (who has my 1150) would probably sing and dance all night.

Well, maybe not-- she'd be too busy downloading and reading manga. ;)

nrapallo

09-06-2008, 03:18 PM

Excllent. Now if someone could just get .cbz downloads working, my younger daughter (who has my 1150) would probably sing and dance all night.

Well, maybe not-- she'd be too busy downloading and reading manga. ;)

If you have any web references for .cbz that I can test with, it would help. I was planning to do a CBR/CBZ Converter, but thought it wouldn't be useful... :smack:

I just finished a PDF Converter using PDFRead's dos command to facilitate the actual conversion from .pdf to .imp. It works best with smaller sized documents.

I also have finished a DJVU Converter, but it's not easy finding direct .djvu ebooks to download from. Most 'ebooks' are wrapped around a html re-director. I did find some DJVU Editions here (http://www.leoyan.com/djvu-editions.com/filelist4.html) (DjVu BUNDLED) and downloaded Jane Austen's Sense and Sensibility here http://www.leoyan.com/djvu-editions.com/AUSTEN/SENSE/Download.djvu (only had it convert first 5 of 200+ pages though).

While it's nice to have the Converters, they do take a long time and could just as easily be converted offline using PDFRead on the computer and then transferring the resulting .imp, after the conversion, to the ebook reader.

Oh, well, it's good to have a choice... :2thumbsup

p.s. these new plugins will be uploaded shortly...

DaleDe

09-06-2008, 04:11 PM

nrapallo

09-06-2008, 04:26 PM

cbz is really easy. It is just a zip file containing images that are referenced in alphabetical order. The images themselves can be jpg, gif, or png so they should come across ok to be converted for eb-1150 use. I would expect a tool to simply unzip, size the images, and place them in order with each on a page. An option might be to do landscape and split the images in half for larger pages.

Dale

That's exactly what PDFRead does!

The CBZ plugin will just call PDFRead, but PDFRead will create the .imp. This is the same method used for the .pdf and .djvu plugins that will be upload here.

nrapallo

09-06-2008, 05:24 PM

For my latest imperve plugins and a mini tutorial on their use, see impserve: user-created plugins and requests.... (http://www.mobileread.com/forums/showthread.php?t=28863)

Ashish, what a pandora's box you have unleashed .... :eek: :dtw: :party4:

Roberts324

09-09-2008, 01:12 PM

nrapallo

09-09-2008, 01:37 PM

My system won't load PrcConverter.py...:(

The other plugin loads ok, though.

Is there an index.html file available somewhere to help testing all those nice new features?

Bravo to you, ashkulz and others for those nice programs!

Robert:

Are using the windows version of impserve? It currently doesn't load the new plugins that I released (http://www.mobileread.com/forums/showthread.php?t=28863); only uses the ones that came with it. - EDIT: works now in version 0.5!

Ashkulz is looking into it.

A temporary solution would be to install python 2.5 and run the python scripts instead of using the windows version. I got the plugins to work using the python script (run.py) but not the windows version :(.

I think ashkulz may be releasing a new version soon so hopefully this part gets fixed and users can start to enjoy the benefits this program has to offer.

The index.html I use is located in this post http://www.mobileread.com/forums/showthread.php?p=246855#post246855 and is in the attachment called content.zip.

Have fun!

Roberts324

09-09-2008, 04:09 PM

Are using the windows version of impserve? It currently doesn't load the new plugins that I released (http://www.mobileread.com/forums/showthread.php?t=28863); only uses the ones that came with it.
Ashkulz is looking into it.
Fine, I'll just wait.
The index.html I use is located in this post http://www.mobileread.com/forums/showthread.php?p=246855#post246855 and is in the attachment called content.zip.
Good, this is what I have been using so far. Many crashes have occured, but some links work, sort of, including my home page at http://www.astrosurf.com/soubie (http://www.astrosurf.com)

Bug report: I have found improper mapping of accented characters, where a lowcase e acute is represented by an upcase Egrave. Visible e.g. at news.google.fr.

Amitiés, Robert

I should receive my new Asus eee PC 900 tomorrow... :xmas: more programming ahead... Guess I'll have to draw Delphi from the mothballs...

Roberts324

09-09-2008, 04:23 PM

Bug report: I have found improper mapping of accented characters, where a lowcase e acute is represented by an upcase Egrave. Visible e.g. at news.google.fr.

Not sure this is known, I discovered this feature a couple of hours ago; when the EBW1150 hangs with a data fault and a register dump, if one presses the on/off switch for a long time, a "Reset" button may appear, superimposed on the dump; clicking this prevents you to have to use the reset switch. Does not work allways, though.

nrapallo

09-09-2008, 05:06 PM

Not sure this is known, I discovered this feature a couple of hours ago; when the EBW1150 hangs with a data fault and a register dump, if one presses the on/off switch for a long time, a "Reset" button may appear, superimposed on the dump; clicking this prevents you to have to use the reset switch. Does not work allways, though.

Actually, you don't need to hold the on/off button; the "reset" button will appear anyways after 10-15 sec. (But, if you WANT to hold the on/off button, that's OK too... :))

See my proposed solution to this problem here (http://www.mobileread.com/forums/showthread.php?p=248981#post248981).

ashkulz

09-09-2008, 06:49 PM

I've released ebookutils-0.5 at the new site http://ebookutils.berlios.de As it's based on a Wiki, you are free to edit it if you feel you can add something :)

For my latest imperve plugins and a mini tutorial on their use, see impserve: user-created plugins and requests.... (http://www.mobileread.com/forums/showthread.php?t=28863)

Ashish, what a pandora's box you have unleashed .... :eek: :dtw: :party4: Well, I hoped for it and I got it so I'm happy :thumbsup:

Robert:

Are using the windows version of impserve? It currently doesn't load the new plugins that I released (http://www.mobileread.com/forums/showthread.php?t=28863); only uses the ones that came with it. Fixed in 0.5

nrapallo

09-09-2008, 09:52 PM

I've released ebookutils-0.5 at the new site http://ebookutils.berlios.de As it's based on a Wiki, you are free to edit it if you feel you can add something :)

Well, I hoped for it and I got it so I'm happy :thumbsup:

Fixed in 0.5

Ashish:

:chinscratch: Let's see, you set these lofty goals and you attain them. Must be a pretty good feeling! :thumbsup:

Not everyone can wake up one morning and think "I should allow my reader to retrieve (be served) local content as well as surf the internet and download from there in under a minute." Yeah, I almost had the same thought, not! :rofl:

BTW, I can attest that the windows version 0.5 now works with my plugins, except #7 AdjustImageSize which requires PIL to be installed (and compiled therein). Oh, oh, Ashish, could you include PIL's Image?

I do note, however, that the EBW1150 does crash a lot, so I think it's best for now to surf the (friendly) pages of the mobile edition of mobileread, namely: www.mobileread.mobi !

Happy .imp downloading (and .prc/.pdf converting "on-the-fly"). Any suggestions for (plugin) improvements?

Roberts324

09-10-2008, 04:54 AM

I've released ebookutils-0.5 at the new site http://ebookutils.berlios.de As it's based on a Wiki, you are free to edit it if you feel you can add something :)

Well, I hoped for it and I got it so I'm happy :thumbsup:

Fixed in 0.5

Fix confirmed on my system.

A question: when surfing to mobileread's .imp files pages and clicking on a book.imp title, I get a new page. At the bottom of that page is a message: "No Thread specified. If you followed a valid link, please notify the administrator." The .imp file is not downloaded. What am I doing wrong?

ashkulz

09-10-2008, 07:56 AM

nrapallo

09-10-2008, 08:31 AM

Fix confirmed on my system.

A question: when surfing to mobileread's .imp files pages and clicking on a book.imp title, I get a new page. At the bottom of that page is a message: "No Thread specified. If you followed a valid link, please notify the administrator." The .imp file is not downloaded. What am I doing wrong?

Does .prc downloading (and converting) work as well?

Start from my content index.html and click the link to: 'E-Book Uploads: Mobi/PRC Books' and page down until you see the thread for a .prc ebook, say 'Church, R. W.: Bacon, v.1, 6 Sept 2008', After you enter that thread, again page down until you see the underlined attachment, in this case, 'CHURCH_BACON.prc'. When you click that link, the PrcConverter plugin should convert it to .imp and initate a direct download.

As a test, create a next.html and place it in the content directory. In next.html, place your test direct links to .imp or .prc ebooks, like:<html>
<head>
<title>Welcome</title>
</head>
<body>
<h1>Welcome to the second content page</h1>
<p> Go to <A UNDERLINE='YES' HREF="http://www.mobileread.com/forums/attachment.php?attachmentid=16065&d=1220715504">CHURCH_BACON.prc</a></p>
<p> Go to <A UNDERLINE='YES' HREF="http://www.mobileread.com/forums/attachment.php?attachmentid=16079&d=1220752997">The Mind of the Maker.imp</a></p>
</body>
</html>

Then from my index.html, click 'Go to local content next page'

For testing an internet link to a site that uses <form> and plays a variant of the classic kids game 'Rock Paper Scissors', add this link to next.html:<p> Go to <A UNDERLINE='YES' HREF="http://www.penzilla.net/tutorials/python/cgi/rpssl.html">Sample cgi Rpssl page</A> page.</p>

This one is called 'Rock Paper Scissors Spock Lizard'. :rofl:
http://www.penzilla.net/tutorials/python/cgi/rpssl.html

Hope you get this part working soon! This is so much fun.... and some of it IS practical too!

ashkulz

09-15-2008, 05:30 PM

Changes in 0.6:

Only books placed inside the 'shelf' folder will be available by default: if you specify folders on the command line, it will override this location. Also, if an ebook version [1150 vs 1200] has been detected, then only .IMP files matching that ebook will be returned in the online bookshelf.

* setup PyPI upload
* detect ebook version when connecting to authenticate.ebooksystem.net
* filter booklist by ebook type (if detected)
* allow plugins to access ebook type via impserve.config.ebook_type [0 = None, 1 = 1200, 2 = 1150]

shighfield

09-19-2008, 05:37 PM

nrapallo

09-19-2008, 05:44 PM

Any chance you could get the program to write a log file and display nothing on the screen at all?

I've come up with an interesting little wrapper to launch it and minimize to the tray.

If you could create a command line switch to not display anything at all on the screen I could make my little wrapper look pretty nice. <G> Doing it in Virtual Pascal and will release the source code as I do with everything else I code.

Quick way would be to redirect the output to a file (append '>file-log.txt') or to the null device on windows! (append '>nul').

The former still has some of impserve's output going to the screen so maybe the >nul is the way to go for now, that is, until ashkulz chimes in!

ashkulz

09-19-2008, 07:03 PM

Any chance you could get the program to write a log file and display nothing on the screen at all?

I've come up with an interesting little wrapper to launch it and minimize to the tray.

If you could create a command line switch to not display anything at all on the screen I could make my little wrapper look pretty nice. <G> Doing it in Virtual Pascal and will release the source code as I do with everything else I code. I've committed that change (http://git.berlios.de/cgi-bin/gitweb.cgi?p=ebookutils;a=commitdiff;h=9ddd7d023a6 5878286f410d440f03eaff98e93bf). However, the initial banner will still show up on the console. Will that be OK?

Also, we're planning to release 0.7 in a day or two so maybe we could discuss how to integrate your wrapper for that release...

shighfield

09-19-2008, 11:23 PM

The inital banner is fine, as I can compensate for that and place it in it's own "window". It just seems to be the way it's writing to the screen... Not sure why but it would change window sizes etc. I'm /assuming/ it's due to my poor coding ability though. ;)

I do have the wrapper working in that it just hides impserve now completely. I should be able to tweak the code enough by tomorrow so it will hide to the tray in windows so you can quit it with a right click as opposed to the current method of task manager. ;)

If I was writing this just for myself I'd stop now as it works for me... I hate having things output and windows in my taskbar etc. So I assume other's might like to hide it all as well. ;)

When it's working I'll zip up the exe, with the pascal source code and a readme and you could just place it on the wiki page if you wanted? It's not fancy enough to warrant anything other then a plugin placeholder. :)

nrapallo

09-20-2008, 12:23 AM

Using the below sample next.html, will allow you to play with some content-types now better understood by impserve as well as some kindle/mobi sites that list many ebooks. Some of these links may not be operational until the latest changes appear in version 0.7.

You may also require my plugins in the separate impserve plugins thread (http://www.mobileread.com/forums/showthread.php?t=28863).

Have fun!

p.s. be careful to backup your previous next.html or call this one next2.html and link it from your next.html. Uh, did I lose you!

ashkulz

09-20-2008, 04:28 AM

shighfield

09-20-2008, 11:50 AM

Sounds good! I am going to continue with this just so I can finish my learning on how to make a console app minimize to the tray. Now that the pressure is off I'm sure I'll get it done much faster. <G>

ashkulz

09-21-2008, 07:26 AM

Changes in ebookutils-0.7

Add support for running as a windows service
Various fixes to get Munsey's and Feedbooks working (working around limitations in the ebook)
Added a debug option (-d) for detailed logging of proxy responses
Added a log option (-l) to log all output to a file

I'm updating the documentation on the ebookutils wiki right now, and nrapallo will be creating a content package (to get you started) which will also be posted there.

=X=

09-22-2008, 01:33 PM

ashkulz

09-22-2008, 02:09 PM

I'm using the impbuild from the command line and I get the following error.

Note I do have eBook Publisher, Windows XP and downloaded the exe from the web site.

Execution failed at src/impbuild.c:234 while attempting to execute
dhPutValue(impProject, L".ConvertToJPEG = %b", TO_BOOL(image_conv))

Member: .ConvertToJPEG = %b
Function: PutValue
Error In: InvokeArray
Error: Object doesn't support this property or method: 'ConvertToJPEG'
Code: 80020006
Source: IDispatch Interface

Thanks in advance
=X= Please ensure that you have the latest eBook Publisher (2.2.5) installed. Also, if you recently installed GEB Librarian then please (re)install eBook Publisher -- it overrides the DLL registration as they use the same name.

=X=

09-22-2008, 04:39 PM

Please ensure that you have the latest eBook Publisher (2.2.5) installed. Also, if you recently installed GEB Librarian then please (re)install eBook Publisher -- it overrides the DLL registration as they use the same name.

Thanks ashkulz it was GEB that caused the problem. Problem resolved

=X=

nrapallo

09-22-2008, 04:58 PM

Thanks ashkulz it was GEB that caused the problem. Problem resolved

=X=

Well, now, you know which program is going to be sitting on the shelf, so to speak! :rolleyes:

Tells us how you make out exploring impserve! From my point of view, were the plugins easy to install and did they do the job right?

If you get stuck, there may be a work around, given the EBW1150's internal browser is somewhat limited and there are many complex (including poorly coded!) websites out there. We've probably been there, done that, so just ask!

Enjoy!

GeneS

09-22-2008, 08:25 PM

ashkulz

09-22-2008, 11:44 PM

Given all the enthusiasm (particularly comments indicating impserve may be superior to the ebookwise Librarian), I finally talked myself into trying it with the following disappointing results:

impserve -a 127.0.0.1 -p 9090 shelf
Loaded plugin: AdjustHtml
Loaded plugin: AdjustQueryString
impserve 0.7: starting server on 127.0.0.1:9090
127.0.0.1 - - [22/Sep/2008 20:19:00] Error encountered while processing the request:\'NoneType' object has no attribute 'id'

Since my knowledge of Python barely extends beyond the ability to spell it, is anyone able to shed any light on my problem.

Thanks,
Gene GeneS: looks like you have a corrupt .IMP file in your shelf folder. Can you move all the books somewhere else and add it one-by-one to find out which one is causing the problem?

BTW, you don't need to pass in command line parameters: the defaults are good enough, so you can just double click on the EXE.

EDIT: I've committed a fix which should ignore such files. Can you download the latest source code (http://git.berlios.de/cgi-bin/gitweb.cgi?p=ebookutils;a=summary) (click on snapshot - zip) and test it using python src/impserve.py?

=X=

09-23-2008, 01:22 AM

Well, now, you know which program is going to be sitting on the shelf, so to speak! :rolleyes:

Enjoy!
I just tried the application and it is SWEET! My daughters 1150 is now :rohard:

I've just integrated impbuild with my BookCreator tool. This is by far the best IMP creation combo I've found yet.

The contributions ashkulz and you have made are filling in a huge void the 1150 had.
Thanks!

=X=

GeneS

09-23-2008, 08:16 AM

GeneS: looks like you have a corrupt .IMP file in your shelf folder. Can you move all the books somewhere else and add it one-by-one to find out which one is causing the problem?

EDIT: I've committed a fix which should ignore such files. Can you download the latest source code (http://git.berlios.de/cgi-bin/gitweb.cgi?p=ebookutils;a=summary) (click on snapshot - zip) and test it using python src/impserve.py?

Ashkulz:
I now get to see what others are talking about ... Great piece of work. :thumbsup:
I found and removed the corrupt file (it was a 0-byte file). Also tested the fix you provided. Worked like a charm :2thumbsup
Are there any plans to incorporate the fix into the Windows .exe version?

Now, on to trying out some of Nick's plug-ins.

Thanks for the work you and Nick have put into this.

Gene

ashkulz

09-23-2008, 02:36 PM

askyn

09-23-2008, 03:50 PM

quick question, and sorry if its pretty basic.

If im using the batch file to start impserv as a windows service, how do i specify a different directory for my books etc?

thank you.

edit-i r a gud spellar

askyn

09-23-2008, 05:57 PM

nrapallo

09-23-2008, 08:33 PM

Also something strange about the current verions (v8).

I can go to my online book shelf just fine, but to get to the "content" index.html to vist web pages, I have to press the "update now" button.

NOT the "bookstore" option.

If i press bookstore it goes into a infinite loop tring to comunicate with the ebook network.

I can then press the update option and it goes to the index just fine.

Before my crash ( had to restore my computer form scratch ), I could press the bookstore to get online.

Just mentioning it.

The impserve.exe executable, when double clicked, worked for me. :chinscratch:

Try it again after rebooting your reader, perhaps it had a brain-freeze or something... :rolleyes:

BTW, how did you start that session that only allowed the Update Now button to work instead of the directory icon?

askyn

09-23-2008, 11:51 PM

The impserve.exe executable, when double clicked, worked for me. :chinscratch:

Try it again after rebooting your reader, perhaps it had a brain-freeze or something... :rolleyes:

BTW, how did you start that session that only allowed the Update Now button to work instead of the directory icon?

I had started impserv as a windows service at that time.

now that being said, dont you just love flaky people who cannt even reproduce their own error!? rofl

I rebooted my pc and reset my ebook and alls working well now .. i can access content via pressing my directory button etc.

My only remaining question is how to change / add a directory for books to be in when starting impserv as a windows service?

Ive tried a couple of things I thought would work, but it didnt rofl.

Any help / suggestions would be very welcome.

I tried changing this line:

reg add HKLM\SYSTEM\CurrentControlSet\Services\impserve\Pa rameters /v AppDirectory /d "\"%~dp0\"" /f

to add a directory .. should I be modifing this one?

reg add HKLM\SYSTEM\CurrentControlSet\Services\impserve\Pa rameters /v Application /d "\"%~dp0impserve.exe\"" /f

instead? and if so how would I change it?

I want to add in this directory to be searched:
C:\Documents and Settings\Larry\My Documents\Ebooks\Imp format books\

It has multiple directorys with a bunch of books ( each seperate directory is a authors name with their books in it )

nrapallo

09-24-2008, 12:14 AM

Yes, I would like to know as well!

I haven't had much success using the windows service yet. My plugins don't work well in that configuration so I'll be re-vamping them again, if it's possible to get this working!

I just use a batch file for now with my preferred library location (as do you).

Askyn, these are great questions, so keep asking! (he, he, pun intended)

ashkulz

09-24-2008, 01:14 AM

It's pretty simple, you should change this line: reg add HKLM\SYSTEM\CurrentControlSet\Services\impserve\Pa rameters /v AppParameters /d "-l \"%~dp0\impserve.log\"" /ftoreg add HKLM\SYSTEM\CurrentControlSet\Services\impserve\Pa rameters /v AppParameters /d "\"YOURDIRGOESHERE\" -l \"%~dp0\impserve.log\"" /fThe \" is required as the directory in which you install may have spaces...

ashkulz

09-24-2008, 01:16 AM

I haven't had much success using the windows service yet. My plugins don't work well in that configuration so I'll be re-vamping them again, if it's possible to get this working! Well, you wanted to do an overhaul of the plugins anyway, didn't you? So now's your chance :thumbsup:

GeneS

09-24-2008, 09:15 AM

Also something strange about the current verions (v8).
I can go to my online book shelf just fine, but to get to the "content" index.html to vist web pages, I have to press the "update now" button.

NOT the "bookstore" option.

If i press bookstore it goes into a infinite loop tring to comunicate with the ebook network.

I can then press the update option and it goes to the index just fine.

Before my crash ( had to restore my computer form scratch ), I could press the bookstore to get online.

.

Using version 8.
I'm experiencing the same circumstances as Askyn. To access the web, I first have to access the "online bookshelf". Following that, I must press Menu > Update_Now (pressing "Bookstore" creates an infinite search state). If if just go to the menu bar without first accessing my bookshelf, the reader goes into an infinite search for connection regardless of whether I press "Bookstore" or "Update Now." Apparently, the "Update Now" button only works if the reader is already connected to the server.

nrapallo

09-24-2008, 09:22 AM

Changes in ebookutils-0.8:

fix crash when invalid .IMP encountered in bookshelf
added sample content homepage and some sample ebooks

The sample homepage is very minimalistic. Please take care not to overwrite your changes when you upgrade in future!

Here's a replacement (index.html) for those that want some meat & potatoes with their surfing.... :2thumbsup

EDIT: I updated 'index.html' to fix the MR wiki link (it was wrong!).

ashkulz

09-24-2008, 09:26 AM

Using version 8.
I'm experiencing the same circumstances as Askyn. To access the web, I first have to access the "online bookshelf". Following that, I must press Menu > Update_Now (pressing "Bookstore" creates an infinite search state). If if just go to the menu bar without first accessing my bookshelf, the reader goes into an infinite search for connection regardless of whether I press "Bookstore" or "Update Now." Apparently, the "Update Now" button only works if the reader is already connected to the server. Hmm, I'll debug this when I get home. I don't test with my 1150 much, just the 1200 and it works fine for me. Here's a replacement (index.html) for those that want some meat & potatoes with their surfing.... :2thumbsupThat's the point of the minimalistic approach: you can't please everyone so rather than pleasing someone you leave everyone equally dissatisfied and eager to roll out their own ;)

askyn

09-24-2008, 10:32 AM

hmm have another interesting thing happen to me last night.

Im not sure if its a Impserv thing or due to a possible corrupt imp files tho.

Let me explain.

in another posting I had asked for a simple way to edit imp files to update author / title / catagory information so that files I import are organized the way I want and have the title info etc the way i want.

It was recommended that I try Imp Librarian, with the possiblity that it might prevent geb lib from editing them in the future etc. ( I dont use geb lib tho so didnt matter )

Here is the issue tho, after editing all the information and cleaning up titles etc.

Im unable to transfer the files TO my 1150.

I can see them in my online bookshelf just fine, totally organized, etc. I select to download the file, it transfers, then errors out on the 1150 with the following error:

Failed to update book's unique id; report this error (28)

in the impserv window it says this:

impserve 0.8: starting server on 127.0.0.1:9090
127.0.0.1 - - [24/Sep/2008 07:20:28] "GET http://bookshelf.ebooksystem.net/books
helf/default.asp?INDEX=1&REQUEST=100&SHOW_HIDDEN=NO HTTP/1.0" 200 -
127.0.0.1 - - [24/Sep/2008 07:20:30] "GET http://bookshelf.ebooksystem.net/books
helf/getbook?eebook:guid-c9ce0db8493c4dd1a4c91cd526a6a582 HTTP/1.0" 200 -
127.0.0.1 - - [24/Sep/2008 07:20:38] code 404, message File not found
127.0.0.1 - - [24/Sep/2008 07:20:38] "GET http://bookshelf.ebooksystem.net/books
helf/getbook?eebook:guid-c9ce0db8493c4dd1a4c91cd526a6a582 HTTP/1.0" 404 -

help? I have a feeling that the imp librarian has corrupted the files some how ( good thing they are back up copies ! )

nrapallo

09-24-2008, 10:55 AM

help? I have a feeling that the imp librarian has corrupted the files some how ( good thing they are back up copies ! )

Thank you, thank you, for doing this on backups! I thought it might have happened just to me. Good thing you read my WARNING!

I have a way to dissect a .imp so I'll do some more thorough testing of this issue, later today, when I get home from work. Yeah, I do work... :smack:

Would it be possible to upload here one of those corrupt .imp files in the meantime? I mean a non-copyrighted one!

BTW, I don't think it's impserve's issue.

askyn

09-24-2008, 02:04 PM

nrapallo

09-24-2008, 02:09 PM

Here's a replacement (index.html) for those that want some meat & potatoes with their surfing.... :2thumbsup

EDIT: I updated 'index.html' to fix the MR wiki link (it was wrong!).

Ditto!

nrapallo

09-24-2008, 02:17 PM

Sure thing !

And thank you for the warning =)

ive been working with only back upcopies so far any way. But this is another reason to be carefull !!

Here is one of the files that doesnt load:

Haggard, H. Rider - SHE

currently free from Project Gutenberg, altho i got my current copy from a different web site.

Yep, it's a problem with Imp Librarian!!!! See attached .zip for some goodies...

Basically the .RES directory name appears as "ocket - She.RESBAHKS", which is 5 characters offset. It should probably have been "Mobipocket - She.RES"!

OOH and just as a fyi, I emailed the author of Imp Librarian, very fast response but basically said they did this program a long time ago, never made money off of it and there is no support.

Oh, that's sad :( news! He obviously had some nice ideas... :2thumbsup

shighfield

09-24-2008, 06:50 PM

We'll just have to keep harassing nick for a new program to change categories. :) I also have a storage system, and within that structure I use categories quite extensively.

Since I can't load /all/ my books onto the device at once, I have to use some categories with numbers so I know if I have read the book, and when I read the book I edit the title and add the date there.

Huge pain, but I am so scatterbrained I need some type of system. I just wish I had the time to devote to coding now that I did in the BBS days. ;)

nrapallo

09-24-2008, 07:20 PM

We'll just have to keep harassing nick for a new program to change categories. :)

It's Ok, I'll just harass ashkulz! :eek: :rofl:

Roberts324

10-17-2008, 08:09 AM

It's Ok, I'll just harass ashkulz! :eek: :rofl:

Pleade do :p (as you know, this is a function that _does not_ work on my system with GEBLibrarian, though my configuration seems to the same as yours).

Congratulations to you all, I noticed a lot of improvements on the last versions of impserve, indeed.

I just tried the web surfing functions and found some - minor but often encountered - problems with accented letters as are used in French and many other languages; here is a way to check this point :

From the "Welcome to Impserve" page on the EBW, go to my Lulu storefront (books I translate and publish) at www.lulu.com/robert_soubie. You will notice that words with accented characters in them are not correctly rendered ; for instant, "inédit" comes out with a square root and a copyrigth sign in lieu of the "é".

Of course this has to be an encoding problem (UTF-8 vs ???).

PS: This message was at first ill-placed, so I copied it here.

nrapallo

10-25-2008, 08:20 AM

Using version 8.
I'm experiencing the same circumstances as Askyn. To access the web, I first have to access the "online bookshelf". Following that, I must press Menu > Update_Now (pressing "Bookstore" creates an infinite search state). If if just go to the menu bar without first accessing my bookshelf, the reader goes into an infinite search for connection regardless of whether I press "Bookstore" or "Update Now." Apparently, the "Update Now" button only works if the reader is already connected to the server.

I just experienced this on my EBW1150!

After the 'Online Bookshelf' tab is tapped, I could not use the 'Bookstore' menu item (infinite searching...). The only thing that worked was the 'Update Now' menu item. Exactly like above! :angry:

To fix this, I changed the reader's Phone Connection settings to not 'Use HTTP proxy' nor 'Use proxy for USB'. Then I connected to the bookshelf and let my reader 'call home' (the ETI servers). I explorered the choices there a bit and then returned to my original reader Phone Connection settings (checked those two choices again).

Now, everything works "as advertised"!

I attach before and after logs to see the different situations.

Roberts324

10-26-2008, 06:48 AM

Hi Nick,

Any idea on the accented letetrs problem I mentionned above?

nrapallo

10-26-2008, 01:04 PM

Pleade do :p (as you know, this is a function that _does not_ work on my system with GEBLibrarian, though my configuration seems to the same as yours).

Congratulations to you all, I noticed a lot of improvements on the last versions of impserve, indeed.

I just tried the web surfing functions and found some - minor but often encountered - problems with accented letters as are used in French and many other languages; here is a way to check this point :

From the "Welcome to Impserve" page on the EBW, go to my Lulu storefront (books I translate and publish) at www.lulu.com/robert_soubie. You will notice that words with accented characters in them are not correctly rendered ; for instant, "inédit" comes out with a square root and a copyrigth sign in lieu of the "é".

Of course this has to be an encoding problem (UTF-8 vs ???).

PS: This message was at first ill-placed, so I copied it here.

Yes, it's a UTF-8 encoding issue. The text in your webpage is in UTF-8 encoding (ie. two characters to represent e acute, but the native ebookwise reader is happy to render é as é in windows-1252 encoding.

Can you replace the é in UTF-8 with a é as é ?

Partial website text in windows-1252:Le roman Le Nouvel Adam, inédit en France, vient de paraître ; comme les autres oeuvres de Stanley G. Weinbaum, il est disponible en version brochée et, pour les collectionneurs et les bibliothèques de prêt, en version reliée avec jaquette en quadrichromie anti-poussière.

Partial website text with tidy in UTF-8:Le roman <i>Le Nouvel Adam</i>, inÃ©dit en France, vient de paraÃ®tre ; comme les autres oeuvres de Stanley G. Weinbaum, il est disponible en version brochÃ©e et, pour les collectionneurs et les bibliothÃ¨ques de prÃªt, en version reliÃ©e avec jaquette en quadrichromie anti-poussiÃ¨re.</h4>

Partial website text with tidy in window-1252Le roman Le Nouvel Adam, inédit en France, vient de paraître ; comme les autres oeuvres de Stanley G. Weinbaum, il est disponible en version brochée et, pour les collectionneurs et les bibliothèques de prêt, en version reliée avec jaquette en quadrichromie anti-poussière.

Hope this helps!