"Error communicating with device"

[Uncle Robin]

I'm trying to copy my unread books from my Calibre library to my new Sage. There are only 11147, but the transfer keeps failing with
"Error communicating with device"

After this happened twice, I thought it might be a problem with the non-Kobo cable. So I got the supplied cable out of the box and used it. Tried again, same result. Then I wondered if there was a problem with the USB port on my PC, so tried another. Same result. Here's the log of the most recent attempt. The "no such file" book is in my Calibre library.

calibre, version 5.31.1
ERROR: Error: Error communicating with device

[Errno 2] No such file or directory: 'C:\\Users\\username\\AppData\\Local\\Temp\\calibr e__vs85hwl\\83iumcgt_epub_container\\OEBPS\\Fenich el_-_A_Lady%27s_Virtue.xhtml'

Traceback (most recent call last):
File "calibre\gui2\device.py", line 87, in run
File "calibre\gui2\device.py", line 593, in _upload_books
File "calibre_plugins.kobotouch_extended.device.driver" , line 416, in upload_books
File "calibre\devices\kobo\driver.py", line 2143, in upload_books
File "calibre_plugins.kobotouch_extended.device.driver" , line 211, in _modify_epub
File "calibre_plugins.kobotouch_extended.container" , line 119, in __init__
File "calibre_plugins.kobotouch_extended.container" , line 375, in __run_async_over_content
File "calibre_plugins.kobotouch_extended.container" , line 368, in __run_async
File "concurrent\futures\_base.py", line 432, in result
File "concurrent\futures\_base.py", line 388, in __get_result
File "concurrent\futures\thread.py", line 57, in run
File "calibre_plugins.kobotouch_extended.container" , line 291, in forced_cleanup
File "calibre\ebooks\oeb\polish\container.py", line 614, in raw_data
File "calibre\ebooks\oeb\polish\container.py", line 1070, in open
FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\username\\AppData\\Local\\Temp\\calibr e__vs85hwl\\83iumcgt_epub_container\\OEBPS\\Fenich el_-_A_Lady%27s_Virtue.xhtml'

What do I need to do to fix this?

[Uncle Robin]

It turns out that the problem is with that specific book. I can open and read the epub just fine in the Calibre viewer, but I can't send it to my Sage, I keep getting the error message posted above. I've tried deleting it and creating a new epub from another copy, same result.

[Uncle Robin]

Originally Posted by Uncle Robin

It turns out that the problem is with that specific book. I can open and read the epub just fine in the Calibre viewer, but I can't send it to my Sage, I keep getting the error message posted above. I've tried deleting it and creating a new epub from another copy, same result.

I found a workaround - create a kepub from the epub, delete the epub, create an epub from the kepub. That epub I could then send to my device. For some reason, "send to device" only offers epubs, not kepubs. This is not normally an issue as I prefer reading epubs in KOReader, but for this one book I thought I should send the kepub. Which turned out not to be an option.

[davidfor]

The error was happening when KoboTouchExtended driver was doing the work to transform the epub to a kepub. There was probably an error in the epub. An epub-to-epub conversion will frequently fix these errors. And a conversion to kepub, is actually a conversion to epub followed by adding the kepub specific things. That step is identical to what the KoboTouchExtended driver does. As that step didn't fail, it means the epub-to-epub part of the conversion fixed whatever was wrong with the epub. If you still have the original epub, you could open it in the editor and run the error check to see what it reports.

And for the record, the conversion from kepub is basically, remove the spans and then do a conversion to epub. So, you have actually done two epub-to-epub conversions.

The way the drivers work is to look send the format that is at highest in the list of supported formats for the device. For Kobo ereaders, this defaults to kepub first and then epub. That means if you send a book, if you have a kepub in the library, it will be sent. If not, then the epub will be sent. If no supported formats are found, the book will be converted to the first format in the supported list for which you have a conversion output plugin. That means kepub if you have the Kepub Output plugin installed, or epub if you don't.

The KoboTouchExtended driver works like this. It will send a kepub if one exists. Otherwise it sends the epub. But, during the send it transforms the epub to kepub and then sends that over. As far as calibre is concerned, the epub is sent, but, the result on the device is a kepub. Some of the other drivers make changes, but, I think the KoboTouchExtended driver is the one that makes the most changes.

[Uncle Robin]

Originally Posted by davidfor

The error was happening when KoboTouchExtended driver was doing the work to transform the epub to a kepub. There was probably an error in the epub. An epub-to-epub conversion will frequently fix these errors. And a conversion to kepub, is actually a conversion to epub followed by adding the kepub specific things. That step is identical to what the KoboTouchExtended driver does. As that step didn't fail, it means the epub-to-epub part of the conversion fixed whatever was wrong with the epub. If you still have the original epub, you could open it in the editor and run the error check to see what it reports.

And for the record, the conversion from kepub is basically, remove the spans and then do a conversion to epub. So, you have actually done two epub-to-epub conversions.

The way the drivers work is to look send the format that is at highest in the list of supported formats for the device. For Kobo ereaders, this defaults to kepub first and then epub. That means if you send a book, if you have a kepub in the library, it will be sent. If not, then the epub will be sent. If no supported formats are found, the book will be converted to the first format in the supported list for which you have a conversion output plugin. That means kepub if you have the Kepub Output plugin installed, or epub if you don't.

The KoboTouchExtended driver works like this. It will send a kepub if one exists. Otherwise it sends the epub. But, during the send it transforms the epub to kepub and then sends that over. As far as calibre is concerned, the epub is sent, but, the result on the device is a kepub. Some of the other drivers make changes, but, I think the KoboTouchExtended driver is the one that makes the most changes.

Thanks for the explanation. It turned out to be the first of around 6 books with similar issues, making the transfer a tedious process. Happily it's all done now.

[Uncle Robin]

Originally Posted by davidfor

As far as calibre is concerned, the epub is sent, but, the result on the device is a kepub.

One thing that does confuse me a little is why I have exactly zero files on my Sage with the kepub file extension - all 1200 or so are *.epub when viewed in a file explorer? "Enable Extended Kobo Features" is unchecked in the Driver settings, which I thought would have meant epubs got transferred without conversion. I guess that's why I'm a bit puzzled by the failure of a few to be transferred

[ownedbycats]

Originally Posted by Uncle Robin

One thing that does confuse me a little is why I have exactly zero files on my Sage with the kepub file extension - all 1200 or so are *.epub when viewed in a file explorer?

By default, Windows Explorer has file extensions hidden. IIRC, the file is actually book.epub.kepub. You sure it's not the last bit being hidden?

[Uncle Robin]

Originally Posted by ownedbycats

By default, Windows Explorer has file extensions hidden.

Yes, that is the default and it's also one of the first things I change on any Windows install - all extensions show on my system. A quick check confirms this a search for *.kepub" returns over 1300 files on my PC, all in Calibre directory. So the files on my Sage are not kepubs

[davidfor]

Originally Posted by Uncle Robin

Yes, that is the default and it's also one of the first things I change on any Windows install - all extensions show on my system. A quick check confirms this a search for *.kepub" returns over 1300 files on my PC, all in Calibre directory. So the files on my Sage are not kepubs

Sideloaded kepubs have the double extension of ".kepub.epub". Purchased kepubs downloaded to the device have no extension. Calibre is pretty much the only place where the extension ".kepub" means anything. And that means you have been converting books to kepubs and keeping them in your library. When you send these books to the device, the kepub format will be sent (assuming you haven't changed from the defaults).

And as I said, there is probably an error in the epubs that the extended driver cannot handle when doing the transformation to kepub. That's was an explicit design decision. It's likely that the epub won't work properly on the device.

[Uncle Robin]

Originally Posted by davidfor

Sideloaded kepubs have the double extension of ".kepub.epub". Purchased kepubs downloaded to the device have no extension. Calibre is pretty much the only place where the extension ".kepub" means anything.

I have no purchased books on my Sage, and no files with ".kepub.epub", only ".epub", 1179 of them.

[PeterT]

@davidfor I'm probably wrong here but I thought the extended driver did not save the kepub file in calibre.

In fact from the project description on GitHub

KoboTouchExtended: This is a device driver plugin extending the capabilities of the existing*KoboTouch*driver provided with Calibre to enable extra Kobo features.*Please note: modified books are not saved back into your Calibre library to prevent issues with overwriting library configurations, so books get converted in transit to your device every time. In general, the extra time required should not be noticeable.

Sent from my Pixel 4a using Tapatalk

[davidfor]

Originally Posted by Uncle Robin

I have no purchased books on my Sage, and no files with ".kepub.epub", only ".epub", 1179 of them.

Then why were you using the KoboTouchExtended driver? It's purpose is to send books as kepubs. If you don't use that, then there is little benefit to using it. From the original error, it is doing some changes to the books that are forced when the extended driver is used. They probably benefit the change to kepub, but, not at other times. Do you have any of the other extended options (ones on the tab "Extended" in the driver configuration) turned on?

[davidfor]

Originally Posted by PeterT

@davidfor I'm probably wrong here but I thought the extended driver did not save the kepub file in calibre.

In fact from the project description on GitHub

Correct, the driver doesn't save the kepubs to the library. But, there is an option to save them somewhere. But, @Uncle Robin has stated he has over 1300 files with the extension .kepub in the calibre directory. They have to have gotten their somewhere. Maybe them being saved in the library directory, but, that would mean they are not showing in the library. Otherwise, they would have to be converted or added somehow and they should show in the library. And they also should get sent to the device as that is the default configuration. But, that can of course be changed.

[Uncle Robin]

Originally Posted by davidfor

Then why were you using the KoboTouchExtended driver?

In a word, inertia. I used to convert all my pubs to kepubs before learning that the Extended Driver would do it for me. Then, when I started using KOReader and found that it seemed to handle epubs better than kepubs, I stopped converting and stopped using the Extended Touch Driver's features. The other reason I keep using it is because I occasionally sideload kepubs to my wife's H2O2

[davidfor]

Originally Posted by Uncle Robin

In a word, inertia. I used to convert all my pubs to kepubs before learning that the Extended Driver would do it for me. Then, when I started using KOReader and found that it seemed to handle epubs better than kepubs, I stopped converting and stopped using the Extended Touch Driver's features. The other reason I keep using it is because I occasionally sideload kepubs to my wife's H2O2

In that case, I'd suggest going back to the KoboTouch driver for your use. For the occasional update to your wife's device, swap to the extended driver. My Kobo Utilities plugin has a way to do that without digging through the calibre preferences.