Error message when sending to device (Kobo Glo HD)

[rosalita]

Today I tried to send a number of EPUB books to my Kobo GloHD. Most of them went without a hitch, but four of them threw up an error message. All of the books that had an error were purchased from Kobo in 2014 and 2016, but many of the ones that did send OK were also.

I've copied the error dialog for one of them here. Any insights on what might be happening and how to fix?


calibre, version 5.23.0
ERROR: Error: Error communicating with device

Failed to process Black Sheep by Georgette Heyer: list index out of range

Traceback (most recent call last):
File "calibre_plugins.kobotouch_extended.device.driver" , line 270, in _modify_epub
common.modify_epub(
File "calibre_plugins.kobotouch_extended.common", line 236, in modify_epub
container.convert()
File "calibre_plugins.kobotouch_extended.container" , line 384, in convert
self.__run_async_over_content(self.add_kobo_spans)
File "calibre_plugins.kobotouch_extended.container" , line 380, in __run_async_over_content
self.__run_async(func, names)
File "calibre_plugins.kobotouch_extended.container" , line 373, in __run_async
future.result(timeout=10)
File "concurrent/futures/_base.py", line 432, in result
File "concurrent/futures/_base.py", line 388, in __get_result
File "concurrent/futures/thread.py", line 57, in run
File "calibre_plugins.kobotouch_extended.container" , line 506, in add_kobo_spans
body = root.xpath("./xhtml:body", namespaces={"xhtml": XHTML_NAMESPACE})[0]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre/gui2/device.py", line 89, in run
File "calibre/gui2/device.py", line 595, in _upload_books
File "calibre_plugins.kobotouch_extended.device.driver" , line 400, in upload_books
return super(KOBOTOUCHEXTENDED, self).upload_books(
File "calibre/devices/kobo/driver.py", line 2137, in upload_books
File "calibre_plugins.kobotouch_extended.device.driver" , line 296, in _modify_epub
raise e.__class__(msg).with_traceback(tb)
File "calibre_plugins.kobotouch_extended.device.driver" , line 270, in _modify_epub
common.modify_epub(
File "calibre_plugins.kobotouch_extended.common", line 236, in modify_epub
container.convert()
File "calibre_plugins.kobotouch_extended.container" , line 384, in convert
self.__run_async_over_content(self.add_kobo_spans)
File "calibre_plugins.kobotouch_extended.container" , line 380, in __run_async_over_content
self.__run_async(func, names)
File "calibre_plugins.kobotouch_extended.container" , line 373, in __run_async
future.result(timeout=10)
File "concurrent/futures/_base.py", line 432, in result
File "concurrent/futures/_base.py", line 388, in __get_result
File "concurrent/futures/thread.py", line 57, in run
File "calibre_plugins.kobotouch_extended.container" , line 506, in add_kobo_spans
body = root.xpath("./xhtml:body", namespaces={"xhtml": XHTML_NAMESPACE})[0]
IndexError: Failed to process Black Sheep by Georgette Heyer: list index out of range

[DNSB]

You might want to check in the Kobo Touch Extended Driver thread. I seem to remember similar errors when the KoboTouchExtended driver was not updated.

[davidfor]

The line numbers do not match the current version of the plugin. As @DNSB implies, you need to update the plugin to the latest version.

[rosalita]

Ah, thank you both! I didn't even notice the "updated plugin available" notice at the bottom of the window — poor form on my part. Regretfully, however, updating the plugin (current version shows as 3.5.3) and restarting Calibre has not solved the problem.

The error dialog:

calibre, version 5.26.0
ERROR: Error: Error communicating with device

Failed to process Frederica by Georgette Heyer: list index out of range

Traceback (most recent call last):
File "calibre_plugins.kobotouch_extended.device.driver" , line 270, in _modify_epub
common.modify_epub(
File "calibre_plugins.kobotouch_extended.common", line 236, in modify_epub
container.convert()
File "calibre_plugins.kobotouch_extended.container" , line 379, in convert
self.__run_async_over_content(self.add_kobo_spans)
File "calibre_plugins.kobotouch_extended.container" , line 375, in __run_async_over_content
self.__run_async(func, names)
File "calibre_plugins.kobotouch_extended.container" , line 368, in __run_async
future.result(timeout=10)
File "concurrent/futures/_base.py", line 432, in result
File "concurrent/futures/_base.py", line 388, in __get_result
File "concurrent/futures/thread.py", line 57, in run
File "calibre_plugins.kobotouch_extended.container" , line 501, in add_kobo_spans
body = root.xpath("./xhtml:body", namespaces={"xhtml": XHTML_NAMESPACE})[0]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre/gui2/device.py", line 89, in run
File "calibre/gui2/device.py", line 595, in _upload_books
File "calibre_plugins.kobotouch_extended.device.driver" , line 400, in upload_books
return super(KOBOTOUCHEXTENDED, self).upload_books(
File "calibre/devices/kobo/driver.py", line 2137, in upload_books
File "calibre_plugins.kobotouch_extended.device.driver" , line 296, in _modify_epub
raise e.__class__(msg).with_traceback(tb)
File "calibre_plugins.kobotouch_extended.device.driver" , line 270, in _modify_epub
common.modify_epub(
File "calibre_plugins.kobotouch_extended.common", line 236, in modify_epub
container.convert()
File "calibre_plugins.kobotouch_extended.container" , line 379, in convert
self.__run_async_over_content(self.add_kobo_spans)
File "calibre_plugins.kobotouch_extended.container" , line 375, in __run_async_over_content
self.__run_async(func, names)
File "calibre_plugins.kobotouch_extended.container" , line 368, in __run_async
future.result(timeout=10)
File "concurrent/futures/_base.py", line 432, in result
File "concurrent/futures/_base.py", line 388, in __get_result
File "concurrent/futures/thread.py", line 57, in run
File "calibre_plugins.kobotouch_extended.container" , line 501, in add_kobo_spans
body = root.xpath("./xhtml:body", namespaces={"xhtml": XHTML_NAMESPACE})[0]
IndexError: Failed to process Frederica by Georgette Heyer: list index out of range

[DNSB]

Does the error occur with epubs other than Frederica being converted to kepub during the transfer to your Kobo?

[davidfor]

The error suggests there is an error in the book code. It looks like one of the internal files has errors and the body tag cannot be found.

I would open the book in the calibre editor and run the error checking tool. That should show where the error is.

[rosalita]

Originally Posted by DNSB

Does the error occur with epubs other than Frederica being converted to kepub during the transfer to your Kobo?

Most books transfer just fine, but there is a set of four, all purchased from Kobo back in 2014 and 2015 and all by Georgette Heyer, that will not transfer. I tried deleting the original files from Calibre and re-downloading in case they had somehow become corrupted, but I get the same error for all three. (Other books have continued to transfer without a problem before, during, and after this occurred).

[rosalita]

Originally Posted by davidfor

The error suggests there is an error in the book code. It looks like one of the internal files has errors and the body tag cannot be found.

I would open the book in the calibre editor and run the error checking tool. That should show where the error is.

Thank you for that suggestion. I will try it tomorrow morning when I'm back at my desk, and report back (hopefully with a successful transfer).

[DNSB]

One possibility is to use calibre to do a epub to epub conversion. I'm not fond of doing this but it does fix most errors in the original epub.

[rosalita]

Originally Posted by rosalita

Thank you for that suggestion. I will try it tomorrow morning when I'm back at my desk, and report back (hopefully with a successful transfer).

I followed David's suggestion to open one of the books in the Calibre book editor.
I'll attach screenshots below, but here's the situation:

1. The Check Book feature returns 1/1 Error, related to an incorrect namespace (attachment 1). I clicked on "Run fix HTML on this file, which will automatically insert the correct namespace."

2. Now it shows 1/40 errors, the top one related to an image being present more than once in the manifest (attachment 2). I clicked "Remove all but the first duplicate item."

3. Now 1/39 errors, and it says "The media type for the Table of Contents must be application/x-dtbncx+xml" related to Line 52 (attachment 3). For reference (this will be relevant later) the code in Line 52 is currently <item href="toc.ncx" id="ncx" media-type="application/xhtml+xml"/>. I clicked "Try to correct all fixable errors automatically".

2. After doing so, the code in Line 52 did indeed change in the way the error message indicated it needed to. But now there's a new error message: "The item OEBPS/toc.ncx present in the spine has the media-type application/x-dtbncx+xml. Most e-book software cannot handle non-HTML spine items. If the item is actually HTML, you should change its media-type to application/xhtml+xml. If it is not-HTML you should consider replacing it with an HTML item, as it is unlikely to work in most readers" (attachment 4).

I have no idea where I'm meant to go with this. Now, I did not try stopping after Step 2 above, saving changes and trying to send the book to my Kobo again. Maybe that would work. But I thought if there were more errors I should try to fix them all, but one of them appears to be some weird Catch-22 that means nothing to me but probably does mean something to you.

I should also add that I tried DNSB's suggestion of doing an EPUB-to-EPUB conversion without editing the book on another of the four, and that did successfully send the book to device. So maybe I should just do that with the other three and call it a day, but I was curious if David or anyone else has insights into what the error messages mean. I am at a deficit when it comes to understanding all but the simplest code, and this one has stumped me thoroughly!

Many thanks for all your help.

[DNSB]

Basically, that ebook is a fine example of how not to create an epub. I'd have to see the entire content.opf file to be more certain but it looks as if the manifest section of the content.opf had the toc.ncx file with the wrong type which was corrected but toc.ncx was also included in the spine section of the content.opf which should never happen. In the lines below, the itemref under the spine for ncx should not be there since it would make the toc.ncx displayable.

    <item id="ncx" href="toc.ncx" media-type="application/x-dtbncx+xml"/>

  <spine toc="ncx">
    <itemref idref="ncx"/>

[rosalita]

Originally Posted by DNSB

Basically, that ebook is a fine example of how not to create an epub.

Goodness! I think I'll just do the EPUB-to-EPUB conversion on the other three and let well enough alone, then.

It's curious, because these books purchased from Kobo have the same publisher and imprint as many other Kobo books from the same author that sent to the device without a peep. I guess they were going through a rough patch with their process but seemed to have worked it out.

Thanks again to you and also davidfor for helping me with this problem. If nothing else, I feel like I have a little more understanding of how to edit an ebook if necessary — time to dig out all those books with highlighted typos to be fixed!

Julia

[PeterT]

I'm just curious as to the exact origin of, for instance, Black Sheep. I know it's from Kobo but then did you download the ePub version from the web site, and strip DRM, or was it the kepub version that had DRM removed?

Sent from my Pixel 4a using Tapatalk

[rosalita]

Originally Posted by PeterT

I'm just curious as to the exact origin of, for instance, Black Sheep. I know it's from Kobo but then did you download the ePub version from the web site, and strip DRM, or was it the kepub version that had DRM removed?

Sent from my Pixel 4a using Tapatalk

I downloaded the ePub from Kobo and imported it into Calibre via ADE. I was able to open it My understanding is that the DRM stripping happens when they are imported, and indeed I was able to open them in Calibre without problem. I do use the plugin that converts to kepub as they are sent to device but I don't think that's the problem, as doing the ePub-to-ePub conversion in Calibre allowed the test book to be transferred to the device as kepub without an issue.

And the downloads from Kobo happened just this past week. When I encountered problems sending the previous versions in Calibre to my device, I deleted those and re-downloaded, thinking they had been corrupted. But the new downloads had the same issue.

[PeterT]

Thanks. I hope to experiment myself shortly

Sent from my Pixel 4a using Tapatalk