Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 01-14-2010, 09:24 AM   #1
elvenic
Enthusiast
elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.
 
Posts: 28
Karma: 16604
Join Date: Dec 2009
Location: San Jose, CA, USA
Device: Kindle PW, Kobo Glo, Kindle 4 Touch, Sony T1
Calibre Globe and Mail recipe and Sony PRS-600

When I download Globe and Mail feed using Calibre recipe and then download it to my Sony PRS-600 and then view it, there is always a page on which the PRS-600 freezes for very long time, even 'reset' does not help immediately, only after couple of minutes of being frozen if I try to repeat 'reset' and power-on the device would reset and come back alive.

I tried to modify the recipe and use if from the command line (ebook-convert) and look at the downloaded HTML to figure out what exactly in the Globe and Mail HTML freezes the PRS-600, but without success.

Any help with this?

Also, when I download generated epub to the Sony PRS-600, Calibre reports following error:

ERROR: ERROR: Unhandled exception: <b>AttributeError</b>:'NoneType' object has no attribute 'split'

Traceback (most recent call last):
File "/home/kovid/build/calibre/src/calibre/gui2/__init__.py", line 245, in dispatch
File "/home/kovid/build/calibre/src/calibre/gui2/device.py", line 902, in books_uploaded
File "/home/kovid/build/calibre/src/calibre/gui2/device.py", line 258, in add_books_to_metadata
File "/home/kovid/build/calibre/src/calibre/devices/prs505/driver.py", line 168, in add_books_to_metadata
AttributeError: 'NoneType' object has no attribute 'split'

Last edited by elvenic; 01-14-2010 at 09:41 AM.
elvenic is offline   Reply With Quote
Old 01-14-2010, 12:24 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,438
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Regrading the error. Did you customize the sony reader and remove all fields for creating collections?

If you can post a link to/attach the html that causes problems, I may have a few suggestions
kovidgoyal is offline   Reply With Quote
Old 01-14-2010, 01:17 PM   #3
elvenic
Enthusiast
elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.
 
Posts: 28
Karma: 16604
Join Date: Dec 2009
Location: San Jose, CA, USA
Device: Kindle PW, Kobo Glo, Kindle 4 Touch, Sony T1
Quote:
Originally Posted by kovidgoyal View Post
Regrading the error. Did you customize the sony reader and remove all fields for creating collections?
No, I did not do anything to the Sony Reader, and I don't even know what are "fields for creating collections"


Quote:
Originally Posted by kovidgoyal View Post
If you can post a link to/attach the html that causes problems, I may have a few suggestions
The 'link' is in the embedded 'Globe and Mail' recipe: just use 'download now', it will generate Globe and Mail epub in the Calibre's list of books; then if you connect Sony PRS-600 to the computer via USB, Calible will download the epub to the Sony reader. Disconnect the USB, when Sony wakes up from the USB stupor, open the Globe and Mail book on it and start flipping pages. You may flip through couple of first articles without a problem, but very soon - this always happens to me within first 50 pages - when you press 'next page', the Sony freezes and I have to do 'reset' to bring it back to life (well, may be if I waited another 10 min. the Sony would wake up by itself)

I think that today at night, when I get home, I might be able to come up with a small epub file that causes the reader to freeze: I was playing with the contents of the directory that

ebook-convert globe_and_mail.recipe <dir>

created and was building the epubs from it with

ebook-convert <dir>/index.html ttt.epub

and downloading them to the Sony PRS-600 via the Calibre - I'm just trying to find out which exactly generated HTML page causes the Sony to freeze.
elvenic is offline   Reply With Quote
Old 01-14-2010, 01:28 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,438
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Use

ebook-convert globe_and_mail.recipe .epub -vv --debug-pipeline p --extract-to x

html from various stages of the download and conversion process will be saved in p and the epub will be unzipped into x
kovidgoyal is offline   Reply With Quote
Old 01-14-2010, 11:46 PM   #5
elvenic
Enthusiast
elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.
 
Posts: 28
Karma: 16604
Join Date: Dec 2009
Location: San Jose, CA, USA
Device: Kindle PW, Kobo Glo, Kindle 4 Touch, Sony T1
OK, first strange results.

I have attached two files:

ttt.tar.gz - contains a directory 'ttt' that contains a file hierarchy with html and other files from which a ebub is built,

ttt.epub - the epub that was built from the directory 'ttt'

In the directory 'ttt', in the subdirectory 'feed_0', only first two subdirectories ('article_1' and 'article_2') contain index.html files that came from the Globe and Mail feed. All other 'article_*' subdirectories contain dummy index.html files with a simple html text 'Empty Article' (in all these directories, the original index.html files frrom the G&M feed are saved under names 'index.html.saved', so that you can restore the real feed data. I don't think that files with extension '.saved' that are not referenced anywhere in the .html files have any influence on the contents of the resulting epub file.)

The _expected_ result would be that the epub file, when viewed, would show first two articles as from the real G&M feed, and third and subsequent articles with text "Empty Article".

The _observed_ result is that when viewed, the epub file shows first two articles as expected, with the text from the real G&M feed, but third and subsequent articles are shown all as the text of the first article (may be there is a recursion somewhere in the feed?) . This happens both when viewed by the Calibre internal e-book viewer, and when downloaded to the Sony PRS-600. But the Sony does not freeze in this case.

This is a strange result - epub should not contain multiple copies of the first article instead of the articles with "Empty Article" text. May be this is a manifestation of the same error that causes the Sony to freeze on the bigger epub?
Attached Files
File Type: gz ttt.tar.gz (219.0 KB, 109 views)
File Type: epub ttt.epub (194.5 KB, 112 views)
elvenic is offline   Reply With Quote
Old 01-14-2010, 11:53 PM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,438
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Check the <spine> and <manifest> elements in the opf file that will tell you what files are being read.
kovidgoyal is offline   Reply With Quote
Old 01-15-2010, 12:15 AM   #7
elvenic
Enthusiast
elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.
 
Posts: 28
Karma: 16604
Join Date: Dec 2009
Location: San Jose, CA, USA
Device: Kindle PW, Kobo Glo, Kindle 4 Touch, Sony T1
Quote:
Originally Posted by kovidgoyal View Post
Check the <spine> and <manifest> elements in the opf file that will tell you what files are being read.
<spine> and <manifest> point to all the right files: when I removed original 'index.html' files, I substituted them with dummy 'index.html' files that contain syntactically correct html. So everything that <manifest> refers to is still there.

But I have just made another discovery : the original index.html file in the third article directory ('article_2') subdirectory contains strange <img> element (with empty 'src' attribute):

<img src="" width="" height="" alt="" class="calibre15"/>

When this element is present in the epub, Sony freezes when it tries to show third article.

When I commented this out :

<!-- <img src="" width="" height="" alt="" class="calibre15"/> -->

and build epub and download it to the Sony, it did not freeze! Maybe this is a source of this trouble?

Is there a way to make the recipe to ignore <img> elements with empty src?

Or may be this should be hardcoded somewhere in the Calibre source as a basic sanity check: what is a point of an image with no src URL?
elvenic is offline   Reply With Quote
Old 01-15-2010, 12:19 AM   #8
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,438
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Sigh! Seriously who wrote the code for adobe digital editions? I'll add a workaround in the next release.
kovidgoyal is offline   Reply With Quote
Old 01-15-2010, 01:27 AM   #9
elvenic
Enthusiast
elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.
 
Posts: 28
Karma: 16604
Join Date: Dec 2009
Location: San Jose, CA, USA
Device: Kindle PW, Kobo Glo, Kindle 4 Touch, Sony T1
Unfortunately, that's not all of it.

I added more download feeds to the recipe that I was playing with, trying to confirm that we have found the source of the bug. I found a img with empty src - there was only one in all download, and removed this line. Then I build the pub (now it has 250 pages) and downloaded it to the Sony device.

The page with the image that I removed shown OK.

But on the next page Sony froze.

The next page contained several images that were setup as a background images with URLs provided in a stylesheet file, and in the stylesheet file the URLs of these images were pointing to the real web addresses of the Globe and Mail servers.

For example, the index.html file in the article_4 directory contains following link:

Code:
<a href="http://www.theglobeandmail.com/video/teen-girl-violence-on-rise/article1430931/?view=picks" class="calibre47">Teen girl violence on rise</a>
(notice class="calibre47" attribute).

The stylesheet file, ../..stylesheet.css, contains following regarding class 'calibre47':

Code:
.calibre47 {
    background-image: url(http://beta.images.theglobeandmail.com/archive/00433/girl_fight_poll__433794gm-d.jpg)
    }
As you can see, the URL points to the "beta.images.theglobeandmail.com", that Sony PRS-600 can't access because it does not have internet connectivity.

I'm not sure how Calibre processes CSS, but may be it would be possible to filter out all external URLs from it when downloading the feed?

Would it be possible to achieve this by adding some code to the recipe? Any advice?
elvenic is offline   Reply With Quote
Old 01-15-2010, 09:01 AM   #10
elvenic
Enthusiast
elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.
 
Posts: 28
Karma: 16604
Join Date: Dec 2009
Location: San Jose, CA, USA
Device: Kindle PW, Kobo Glo, Kindle 4 Touch, Sony T1
Yes, I confirm that the URLs in the "background-image:" stylesheet element that point to external website were a reeson for the second freese: when I manually removed the link to the ../../stylesheet.css file from the index.html files that wanted to show these background images, build epub and downloaded it to the Sony, I was able to flip through all 250 pages of it without the device freezing.
elvenic is offline   Reply With Quote
Old 01-15-2010, 09:36 AM   #11
elvenic
Enthusiast
elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.
 
Posts: 28
Karma: 16604
Join Date: Dec 2009
Location: San Jose, CA, USA
Device: Kindle PW, Kobo Glo, Kindle 4 Touch, Sony T1
Hm, how do I print debug information from postprocess_html()?

And, even more important question, would it be possible to remove a link to the '../../stylesheet.css' file by overriding postprocess_html()? Or the link to this stylesheet file is generated by the Calible after the postprocess_html() returns?
elvenic is offline   Reply With Quote
Old 01-15-2010, 11:58 AM   #12
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,438
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I just removed the video articles from the globe and mail recipe, that should fix this automatically.
kovidgoyal is offline   Reply With Quote
Old 01-15-2010, 12:58 PM   #13
elvenic
Enthusiast
elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.elvenic 's shirt has a full set of merit badges.
 
Posts: 28
Karma: 16604
Join Date: Dec 2009
Location: San Jose, CA, USA
Device: Kindle PW, Kobo Glo, Kindle 4 Touch, Sony T1
Quote:
Originally Posted by kovidgoyal View Post
I just removed the video articles from the globe and mail recipe, that should fix this automatically.
OK, where can I download the new recipe from? I would like to try it with my Sony PRS-600.

And, the original embedded G&M recipe that I started playing with from was removing ALL 'img' tags from the html - i modified it not to remove images, and other then the problems that I described above (empty 'src' and background images via css, but that is not really about 'img' tag) I did not have any troubles. And having images in the Globe and Mail articles greatly improves reading experience
elvenic is offline   Reply With Quote
Old 01-15-2010, 01:06 PM   #14
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,438
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
If you use the builtin recipe calibre should automatically use the updated recipe. (provided your version of calibre > 0.6.29)
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Sony prs 600 not working after calibre paolo antesitum Calibre 9 09-10-2010 06:50 PM
Calibre and Sony PRS-600 Archergal5219 Calibre 2 07-25-2010 01:32 AM
cALIBRE .71, Sony prs-600 JTAL604622 Calibre 2 06-08-2010 10:23 PM
Help with Boston Globe RSS recipe horsegoalie Calibre 14 12-17-2009 07:56 PM
Calibre and Sony PRS-600 lukasfikr Calibre 10 08-26-2009 10:11 PM


All times are GMT -4. The time now is 02:02 PM.


MobileRead.com is a privately owned, operated and funded community.