[Device Plugin] Kobo Touch Extended Driver

[jgoguen]

Originally Posted by Thalinor

Got it, thank you. I just checked and that book I got directly from the on device bookstore, was in fact a kepub. It did not have a file extension, but when renaming it to zip, I can see the js file you are talking about /js/kobo.js

Good, so you have a correct reference.kepub.epub then. I'm not sure why exactly some kepubs have kobo-android.js, I suspect they're older kepubs that Kobo hasn't updated yet, but this plugin will accept kobo*.js

Originally Posted by Thalinor

Edit: Ok I figure epub's are just container formats (like mp4) which is why the .zip trick worked and files were inside

That's right. The ePub standard is made up of a few related standards, one of which is the Open Container Standard. It's a little more specific than just "a ZIP archive" though; it's required that there be a file named "mimetype" as the first file in the archive, no compression is permitted on this file (but all other files may be compressed) and the file must contain exactly the text "application/epub+zip" and must not have any trailing characters, including no newline characters.

Originally Posted by Thalinor

I just checked all the books that were copied to the Glo via Calibre and the plugin, they all have a "kobo.js" while the originals on my HD do not have that file inside. I assume that's the file the plugin added renaming the one in the reference file "kobo-android.js" to just "kobo.js". If that's the case, the plugin is working correctly and its not something with that "virtual" kepub I used. Does that all sound correct?

That's right, I rename the file to kobo.js and put it at the base of the ePub file when I copy it over.

[Leonatus]

Thanks for the new version!
May I report my experience, for there were droll things happening.
I've got the Kobo Touch, Fw. 2.5.1. Using the Kobo Tweaks.

first, I disable the Device Interfaces 2.0.10 (both of them), with the Reader unplugged and enabled the Extended Pugin. I connected the Reader with the computer and sent three books from Calibre to the SD-card. All were free ebooks, two of them by Gutenberg.
Book A: Without any manipulation by me with Sigil;
Book B: Slight manipulation, such as text indentation and chapter splitting (TOC);
Book C: "Hard" manipulation incl. insertion of drop caps with imported font.

What happened was this: In all cases the bottom and top lines (author/title) showed correctly. TOC also. But:
In A the bookmarks worked completely fine.
In B bookmarks worked, but lead me steadily to the beginning of each chapter that I was reading, even having stopped at an ensuing page.
In C bookmarks didn't work at all; at every opening of the book I see the cover with the remark: 1% read. Same reaction when I bookmarked manually. Drop caps as imported font disappeard, text indentation likewise. Only text jusitification is the same as before.

During the test even once appeared the notion that I had removed the SD-card, but immediately the reader "recovered" from this.

I cannot explain to me the different behaviour in those cases.

[jgoguen]

Book A sounds like you hit the Holy Grail of processing output. Books B and C sounds like you're seeing the remaining issues with adding the Kobo tags. Not sure why it lost the imported font and indentation though.

[John F]

What do I need to do to see the series information on the device?

I tried:

1) Send books (from calibre)
2) eject device
3) plug in device
4) eject device (I'm still in calibre)

The series information doesn't show up.

if I then:

1) plug in device
2) send books
3) eject device

the series information shows up.

I'm using version 0.9.28.

Thanks.

[jgoguen]

Just to make sure, the first time you eject the device after sending books you wait until the Kobo device is done processing the books before you plug it back in?

[John F]

Originally Posted by jgoguen

Just to make sure, the first time you eject the device after sending books you wait until the Kobo device is done processing the books before you plug it back in?

I believe so. I waited until the "processing content" finished, and then waited for the home screen to flash a couple of times.

I just tried another book, same problem.

When you plug the reader back in after the first transfer, how can you tell when processing is done? i waited until the device showed up in Calibre and counted to ten.

[jgoguen]

Originally Posted by John F

When you plug the reader back in after the first transfer, how can you tell when processing is done? i waited until the device showed up in Calibre and counted to ten.

At the bottom right of calibre is some text, normally it says "Jobs: 0". If there are running jobs (like uploading data to the device) the 0 changes to the number of jobs in the queue. If you click on that text, it opens a window with all the jobs calibre has run and their status. If you start calibre and plug in your device, you should expect to see three jobs completed when it's ready to go.

Can you try to reproduce this again and get a debug log? If you still see the same problem where you have to eject and plug in twice before seeing series info, create a new issue on GitHub. Directions for getting a debug log, and for what you need to give me when creating a GitHub issue, are all in the first post of this thread.

[INCSlayer]

since updating to latest version if i have the "enable extended features"
i get this error message:
calibre, version 0.9.30
ERROR: Error: Error communicating with device

Failed to parse 'Storm Front' by 'Jim Butcher' with error: 'character mapping must return integer, None or unicode' (file: calibre_plugins.kobotouch_extended.container, line: 74)

Traceback (most recent call last):
File "site-packages\calibre\gui2\device.py", line 85, in run
File "site-packages\calibre\gui2\device.py", line 551, in _upload_books
File "calibre_plugins.kobotouch_extended.driver", line 286, in upload_books
InvalidEPub: Failed to parse 'Storm Front' by 'Jim Butcher' with error: 'character mapping must return integer, None or unicode' (file: calibre_plugins.kobotouch_extended.container, line: 74)

[John F]

I had trouble accessing Github.

Here is the log after I plugged it in the second time (and there is no series information for the one book I transferred):

calibre Debug log
calibre 0.9.28 isfrozen: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.3
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Starting up...
Started up in 22.90 seconds with 737 books
Job: 1 Get device information started
DeviceJob: 1 Get device information done, calling callback
DeviceJob: 1 Get device information callback returned
Job: 2 Get list of books on device started
DEBUG: 0.0 KoboTouch:books - oncard='None'
DEBUG: 0.0 KoboTouch:books - prefix='None'
DEBUG: 0.0 Kobo device: Kobo Glo
DEBUG: 0.0 Version of driver: (1, 2, 7) Has kepubs: True
DEBUG: 0.0 Version of firmware: (2, 5, 1) Has kepubs: True
DEBUG: 0.0 [u'', False, False, False, False, False, False, False, False, True, False, u'', True, False, False, False, False, False]
DEBUG: 0.0 KoboTouch:books - set_debugging_title to ''
DEBUG: 0.0 KoboTouch:books - length bl=0
DEBUG: 0.1 KoboTouch:books - length bl after sync=19
Job: 1 Get device information finished
No details available.
DEBUG: 1.1 KoboTouch:books - reading device database
DEBUG: 1.1 Database Version=77
DEBUG: 1.1 KoboTouch:books - shelf list: [u'Discworld', u'Connelly']
DEBUG: 1.1 KoboTouch:books - query= select Title, Attribution, DateCreated, ContentID, MimeType, ContentType, ImageID, ReadStatus, ___ExpirationStatus, FavouritesIndex, Accessibility, IsDownloaded, Series, SeriesNumber, ___UserID from content where BookID is Null and ((Accessibility = -1 and IsDownloaded in ('true', 1)) or (Accessibility in (1,2)) ) and not ((___ExpirationStatus=3 or ___ExpirationStatus is Null) )
DEBUG: 1.3 KoboTouch:books - about to sync_booklists
DEBUG: 1.3 USBMS: starting sync_booklists
DEBUG: 1.5 USBMS: finished sync_booklists
DEBUG: 1.5 KoboTouch:books - have done sync_booklists
DEBUG: 1.5 KoboTouch:books - end - oncard='None'
DEBUG: 1.5 KoboTouch:books - oncard='carda'
DEBUG: 1.5 KoboTouch:books - Asked to process 'carda', but do not have one!
DEBUG: 1.5 KoboTouch:books - oncard='cardb'
DEBUG: 1.5 KoboTouch:books - Asked to process 'cardb', but do not have one!
DeviceJob: 2 Get list of books on device done, calling callback
DeviceJob: 2 Get list of books on device callback returned
Job: 2 Get list of books on device finished
No details available.

This is on a Kobo Glo 2.5.1.

The log is from the second time I plugged in the book. When I looked at the job log, only two jobs were executed.

[jgoguen]

So, I see a few things wrong with this.

First, you're using an old version of the driver. Please upgrade and try again.

Second, I need you to reproduce the issue. You'll need to do at least the following:

• Remove at least 3 of the affected books from your device (so I can see a variety of logs adding books and setting series info later)
• Eject and unplug your device
• Restart calibre in debug mode
• Plug in your device
• Add the books you just removed
• Eject and unplug your device
• Wait for the device to process the files
• Plug in your device
• Wait for calibre to finish its processing (wait for the jobs, in your case two of them, to be complete)
• Eject and unplug your device
• Check for series data
• If there's no series data then plug in your device, let calibre do its processing, eject and unplug your device, and check for series data again
• When series data starts showing up, paste the debug log and let us know how many times it took before you saw series data

Hopefully we won't need you to do more than that but this is what I need to see where the problem may be.

[John F]

Originally Posted by jgoguen

So, I see a few things wrong with this.

First, you're using an old version of the driver. Please upgrade and try again.

...

I'll assume it is fixed in the latest version and check the next time I upgrade Calibre. Thanks for the help.

[riedje]

Originally Posted by INCSlayer

since updating to latest version if i have the "enable extended features"
i get this error message:
calibre, version 0.9.30
ERROR: Error: Error communicating with device

Failed to parse 'Storm Front' by 'Jim Butcher' with error: 'character mapping must return integer, None or unicode' (file: calibre_plugins.kobotouch_extended.container, line: 74)

Since the upgrade to 1.31 earlier today I have the same problems. I first thought there was something wrong with the files I tried to send.

However, files that were uploaded successfully with the previous version are now failing too, with the same error message (character mapping...). I've even trimmed down the epubs to a simple empty body (and checked it with Sigil), but even such a bare-bones file was rejected.

EDIT: Some more info (if it helps..):
Version: Calibre 0.9.30 (x64/Windows 8)
Plugin: 1.31
Device: Kobo Aura HD (firmware 2.5.1)

[jgoguen]

Originally Posted by INCSlayer

since updating to latest version if i have the "enable extended features"
i get this error message:
calibre, version 0.9.30
ERROR: Error: Error communicating with device

Failed to parse 'Storm Front' by 'Jim Butcher' with error: 'character mapping must return integer, None or unicode' (file: calibre_plugins.kobotouch_extended.container, line: 74)

Traceback (most recent call last):
File "site-packages\calibre\gui2\device.py", line 85, in run
File "site-packages\calibre\gui2\device.py", line 551, in _upload_books
File "calibre_plugins.kobotouch_extended.driver", line 286, in upload_books
InvalidEPub: Failed to parse 'Storm Front' by 'Jim Butcher' with error: 'character mapping must return integer, None or unicode' (file: calibre_plugins.kobotouch_extended.container, line: 74)

Originally Posted by riedje

Since the upgrade to 1.31 earlier today I have the same problems. I first thought there was something wrong with the files I tried to send.

However, files that were uploaded successfully with the previous version are now failing too, with the same error message (character mapping...). I've even trimmed down the epubs to a simple empty body (and checked it with Sigil), but even such a bare-bones file was rejected.

EDIT: Some more info (if it helps..):
Version: Calibre 0.9.30 (x64/Windows 8)
Plugin: 1.31
Device: Kobo Aura HD (firmware 2.5.1)

This is interesting. Can one of you please create a GitHub issue (details of what I need you to include are in the first post) so I remember to work on this?

[friedmandu]

Same issue here as INCSlayer and riedje. Details sent to GitHub.
Hope this is of some help.

[Polynésia]

Firmware : 2.5.2
plugin : 1.31
calibre, version 0.9.30 (x64/Windows 8)

ERREUR : Erreur: Erreur pendant la communication avec le lecteur électronique

Failed to parse 'Double Jeu' by 'Abbott, Jeff' with error: 'character mapping must return integer, None or unicode' (file: calibre_plugins.kobotouch_extended.container, line: 74)

Traceback (most recent call last):
File "site-packages\calibre\gui2\device.py", line 85, in run
File "site-packages\calibre\gui2\device.py", line 551, in _upload_books
File "calibre_plugins.kobotouch_extended.driver", line 286, in upload_books
InvalidEPub: Failed to parse 'Double Jeu' by 'Abbott, Jeff' with error: 'character mapping must return integer, None or unicode' (file: calibre_plugins.kobotouch_extended.container, line: 74)