[Device Plugin] Kobo Touch Extended Driver

[jgoguen]

THIS PLUGIN IS BEING INTERMITTENTLY DEVELOPED! FEATURE REQUESTS AND BUG REPORTS MAY TAKE A FEW DAYS TO BE REPLIED TO.



This plugin enables some additional features for all Kobo devices supported by calibre from the Kobo Touch onwards. ePub books processed with this plugin will be rendered using Kobo's internal kepub renderer instead of the standard ePub renderer.



The main benefits of using the kepub renderer (which some may prefer not to have) are:

• Page numbers show the number of page turns remaining in the current chapter instead of the estimated number of pages for the entire book.
  
  • If you don't want this, there is an option in the driver settings to force full-book page numbers.
    
• Reading statistics (time left in this chapter, time for the next chapter, time to complete the book)
  
• The book title is shown at the top of each page.
  
• The chapter title, if any, is shown at the bottom of each page with the page numbers.
  

You are encouraged to install these two plugins as well, to ensure KePub metadata is correctly processed when sending to your Kobo device:

• KePub Metadata Reader - https://www.mobileread.com/forums/sh...d.php?t=261009
  
• KePub Metadata Writer - https://www.mobileread.com/forums/sh...d.php?t=261010
  

Usage

To use this plugin after installing:

• Connect your Kobo Touch/Glo/Mini device and wait for it to be detected by calibre
  
• Select the book(s) you want to send to your device.
  
• Click the Send to device button.
  

Special Notes

• This plugin requires calibre 2.60.0 or later and is only tested with calibre 3.10.
  
• It is recommended that you disable the built-in KoboTouch driver (but not the built-in Kobo driver) while this plugin is installed. On rare occasions, calibre may select the built-in KoboTouch driver instead of this plugin.
  
• This device plugin enables access to Kobo features that are written with the expectation that only unmodified Kobo-purchased books will be used. Kobo reserves the right to make any change at any time, including changes that will cause problems with books created using this plugin ranging from minor to severe, without notice.
  
• When configuring the plugin, it is not necessary to include the '.kepub' portion of the file name; the plugin will add that automatically.
  
• Modified ePub books are not saved to your calibre library in their kepub form; the kepub is only sent to your device, your copy in calibre is never modified.
  

Installation Directions

Open Preferences -> Plugins -> Get new plugins and install the 'Kobo Touch Extended' plugin. You may also download the attached zip file and install the plugin manually, then restart calibre as described in the Introduction to Plugins thread.



Known Issues

Not everything works perfectly all the time. Here are some of the issues I know about that are likely to stick around for a while.

Spoiler:

• When using Kobo firmware 2.9.0 or later, sideloaded KePub files do not display full in-book statistics.
  
• If you have previously installed this plugin in calibre 0.9.18 or earlier, then you upgrade to calibre 0.9.19 or later and can't update the plugin, you must close calibre and manually remove the plugin ZIP file:
  
  • Open the calibre plugin directory.
    
    • Don't know where this is? Before you close calibre, open calibre's preferences, choose Miscellaneous, and click the Open calibre configuration directory button. The plugin directory is in there.
      
  • Delete the file named KoboTouchExtended.zip.
    
  • Start calibre and re-install the plugin.
    
• If you get an error similar to the following:
  
  

  Traceback (most recent call last):
  
  File "site-packages\calibre\gui2\preferences\plugins.py", line 310, in add_plugin
  
  File "site-packages\calibre\customize\ui.py", line 361, in add_plugin
  
  File "site-packages\calibre\customize\ui.py", line 53, in load_plugin
  
  File "site-packages\calibre\customize\zipplugin.py", line 169, in load
  
  File "importlib__init__.py", line 37, in import_module
  
  File "site-packages\calibre\customize\zipplugin.py", line 147, in load_module
  
  File "calibre_plugins.kobotouch_extended.__init__", line 4
  
  ^
  
  SyntaxError: invalid syntax

  The solution is to read the README file from top to bottom
  
• In some circumstances people have reported that while reading a book uploaded with this plugin they suddenly realize that a large amount of time (two hours or more) has passed without their awareness of the passage of time. The only known solution is to finish reading your current book. Voracious readers tend to report this issue more frequently. Please note, sleep is only a temporary resolution.
  

Contributing

Decided you want to contribute to the development of this plugin? Awesome! If you want to contribute to the development of this plugin, you have many options:

• Contribute code, whether for existing bug reports, for new bugs that you found, or for new features that you really want to see implemented. To contribute code, you may fork the repository and send a BitBucket pull request, or you may send me a PM here with a Mercurial patch file.
  
• Submit bug reports on BitBucket. BitBucket bug reports are my to-do list for this plugin; any requests anywhere else are likely to get missed and forgotten. Although I'm happy to discuss the plugin here, I may still ask you to create a bug report; this is so I actually remember to investigate your request!
  
• Test pre-released code from BitBucket. Between releases, new code is committed to the BitBucket repository and may be installed using directions provided in the README file. Based on your testing, you may submit bug reports, provide feedback, think of new feature requests, or just generally enjoy early access (and not necessarily stable code!) to upcoming versions of the plugin.
  
• Translate the plugin on the plugin's Launchpad Translations page.
  

Submitting Bugs

Found an issue? Have a new feature request? Great, I want to hear from you! Go to the BitBucket issue tracker and submit a new bug report. To save us all time, please make sure to include the following details:

• What version of calibre are you running? If this is not the latest version, you will likely be asked to upgrade and confirm that the bug still exists.
  
• What version of the plugin are you using? If this is not the latest version, you will likely be asked to upgrade and confirm that the bug still exists.
  
• What Kobo device you're using and what firmware version
  
  • To get the Kobo firmware version, from the Settings page choose Device Information and look for the line beginning with Software Version about two-thirds of the way down the page.
    
• For issues processing books, whether or not you are able to send me a book that demonstrates the problem. Please be aware of the directions provided when submitting a new bug which specify not to upload a book to the bug report!
  
• The calibre debug log showing the issue, where applicable
  
  • To get the calibre debug log, select Preferences -> Restart in debug mode and reproduce the issue. The calibre debug log will be displayed shortly after closing calibre.
    

Although I'm generally perfectly happy to discuss possible issues here, any issues sent to me through any means other than the Launchpad bug tracker (including email, PM, etc.) may be silently ignored. This is not because I don't care! It's because I have a terrible memory, the BitBucket issue tracker is my "to-do list" for this plugin, and there's been instances where I've seen a post or PM here and promptly forgotten all about it until I happened to stumble across the post again later. If you aren't sure if you have an issue, feel free to post here and we can work together to decide if you need to make a new Launchpad issue or not. If you are sure you have an issue, or if you have an error or exception from calibre, post a new issue. It isn't necessary to post here when you make a new issue, I get notified of all new bugs and I check the list regularly. The worst that could possibly happen on BitBucket is the same that could happen here: I'll tell you that it's not a bug and you'll have the opportunity to convince me otherwise!



README file

The current version is always here on BitBucket.



Version History

Spoiler:

Version 3.0.2 - 2017-12-24
Call load_translations where there are translated strings
Version 3.0.1 - 2017-12-03
Fixed an error uploading DRM-encumbered files. Added new translations.

Version 3.0.0 - 2017-10-30

Imported to BitBucket, made small bugfix and formatting changes, new translations

Version 2.8.2 - 2016-09-19

Fix a typo resulting in an always-true error

Version 2.8.0 - 2016-08-20

Fix a problem with hyphenation introduced in version 2.7.0

Update database code to use apsw for increased performance.

Version 2.7.2 - 2016-07-03

Set minimum calibre version to 2.61.0

Version 2.7.1 - 2016-06-26

Spanish (es_ES) translation update

Version 2.7.0 - 2016-06-26

New settings display format

[B]Version 2.6.2 - 2016-05-16

Fix exceptions raised when the spine has files not actually in the book (a.k.a., fixing what the publisher should be fixing)

[B]Version 2.6.1 - 2016-04-13

Fix spaces being removed from some tags

[B]Version 2.6.0 - 2016-04-09

Be more Kobo-like in KePub generation (whitespace between sentences goes at the beginning of the following span)

Version 2.5.3 - 2015-09-30

Fix problem preventing the cover image from being set properly with unusually-set OPF IDs

Version 2.5.2 - 2015-09-23

Fix mangling of file names when saving to the device.

Version 2.5.1 - 2015-08-05

Fix problems with file names not being converted properly

Version 2.5.0 - 2015-08-03

General tweaks and improvements from adding metadata read/write and conversion input

Version 2.3.3 - 2015-02-02

Update style-hacks.css to remove everything except the padding rules

Version 2.3.2 - 2015-01-30

Update to wrap the book text in divs matching Kobo KePubs, with accompanying CSS changes

Version 2.2.19 - 2014-08-13

Fix an issue with text after empty tags being duplicated

Version 2.2.18 - 2014-07-16

Fix an issue with prefs panes not loading properly

Version 2.2.17 - 2014-07-09

Compatibility with upcoming calibre 2

Version 2.2.16 - 2014-05-30

Bug #1324884 - Image tags need to be wrapped in koboSpan tags

Version 2.2.15 - 2014-05-01

Bug #1310967 - properly decode internal content files that are not UTF-8

Version 2.2.14 - 2013-12-20

Translations update

Version 2.2.13 - 2013-11-16

Support Firmware 3.0.0

Version 2.2.12 - 2013-11-08

Bug #1246505 - Stop adding spaces around em-dashes

Add translation support and initial translations

Version 2.2.11 - 2013-10-28

Support database version 89.

Version 2.2.10 - 2013-10-13

Bug #1227733 - Allow choosing the page number format (normal per-chapter or full-book)

Version 2.2.6 - 2013-08-28

Bug #1217488 - Commit container after mainline driver processing

Version 2.2.5 - 2013-08-25

Bug #1216504: Properly handle Processing Instruction and Comment nodes

Version 2.2.4

Bug #1215698 - Fix crashing on first-time plugin configuration

Version 2.2.3 - 2013-08-20

Code restructured to allow sharing as much code as possible with the new conversion output format plugin.

Update to not process books that have already been processed by the conversion output format plugin.

Version 2.1.0 - 2013-08-02

Bug #1205702 - met67 fixed the nested Kobo spans, which appears to resolve highlighting issues.

General updates to support KoboTouch driver changes in calibre 0.9.42

Version 2.0.2 - 2013-07-30

Bug #1206406 - Update hover text for hyphenation

Bug #1206451 - Do not crash when configuring from the Configure This Device option.

Version 2.0.1 - 2013-07-29

Bug #1205976 - Fix crash on non-ePub file upload

Bug #1204730 - Allow easy choice between ePub and KePub upload

Version 2.0.0 - 2013-07-28

Allow the options dialog to be scrollable so people with small screen resolutions don't lose parts of the dialog.

Bug #1202864 - When copying KePub files to a debug directory, use the driver save template to generate the path to copy the book to.

Bug #1197219 - When silently ignoring failed conversions, actually upload the books as ePub instead of teasing the poor end user with suggestive wording.

Add an option to block award tiles from the home screen.

Improve detection of books already converted to KePub to avoid processing a converted book again.

Bug #1204907 - Change hyphenation option to use CSS-based hyphenation. Updated README with new method and directions.

Bug #1205703 - Embed driver details in converted books for improved debugging.

Version 1.5.0 - 2013-07-02

Bug #73 - Fix auto-detection of the ePub cover image

Bug #69 - Add an option to add a DB trigger to disable the ReventBooks tile when processing new books. Applies to side-loaded and Kobo-synced books.

Add an option to add a DB trigger preventing the Kobo analytics DB table from being updated.

Version 1.4.3 - 2013-06-16

Bug #66 - Do not add spaces after punctuation marks where there shouldn't be any.

Bug #68 - Fix issues with multi-line comment nodes being mangled when smartening punctuation.

Version 1.4.2 - 2013-06-09

Support Kobo FW 2.6.1.

Bug #60 - When sanitizing file names, remove all characters prohibited by Windows.

Bug #61 - Parse content files even if they have the wrong content type.

Bug #65 - Fix errors sending books with smarten punctuation enabled.

Do not add the Kobo JavaScript if there is already a kobo*.js file present.

Tighten up logic when detecting cover image without OPF metadata hints.

Do not add Kobo spans to files that already have them.

Force cleanup of invalid markup.

Version 1.4.1 - 2013-05-30

Bug #56 - Remove slashes from names when sending books

Bug #57 - Fix handling of ProcessingInstruction nodes

Bug #58 - Make sure there's actually text to append to

Version 1.4.0 - 2013-05-27

Bugs #48, 52, 53 - Optimize Kobo span generation, significantly reducing file size and DOM complexity.

Bug #54 - Verify that a node has children before trying to reference child nodes.

Bug #55 - Fix issues with adding spaces where there should be none by including the trailing whitespace in the text group regex

Do not consider the semi-colon a full-stop character. It's used for entity encoding.

Version 1.3.3 - 2013-05-22

Bug #42 - Only set the ImageID for books in the current calibre library.

Bug #43 - Allow specifying a directory to copy generated KePub files to (for debugging purposes).

Bug #44 - Don't process ProcessingInstruction nodes like normal nodes, they're special.

Bug #45 - Fix missing text between HTML tags.

Bug #46 - Update ImageID fields in batches to avoid Python query limits.

Bug #47 - Properly account for multi-line text nodes that start with line-break characters.

Bug #50 - Only set the ImageID field when covers are being uploaded.

Fix issues smartening punctuation when HTML comments are present.

Version 1.3.2 - 2013-05-17

Add significantly more Kobo span tags to improve position detection and hopefully fix bookmarks and annotations

Fix issue preventing this plugin from working with calibre 0.9.30 or later on Windows and Mac due to a bug in Python 2.7.4 (Linux users were unaffected)

Version 1.3.1 - 2013-05-09

Firmware 2.5.2 is now supported

Version 1.3.0 - 2013-05-08

Firmware 2.5.1 is now supported

A new option to smarten punctuation during conversion is now available

Spaces are properly added where text is followed by XHTML tags

Error position reporting is improved

Extract Kobo JavaScript from a provided KePub (see the README for details)

Issues resolved detecting where to add Kobo span tags

Version 1.2.7 - 2013-04-17

Fix an issue where the selected node does not always have a parent to be removed from.

Fix an issue where hyphenating removes spaces when HTML tags follow text.

Version 1.2.6 - 2013-04-17

Fix an issue which prevented non-ePub files from being sent to the device

Add progress reporting

Previous versions - Please see the GitHub commit log

[ShellShock]

thanks. When using this plugin, is the title bar displayed when reading, as usual with kepubs but not with epubs?

[jgoguen]

The title bar is another kepub feature. I don't believe I can disable it.

[buffaloseven]

So just out of curiosity, Calibre plugins are entirely python-based (just making sure I'm not missing anything)? If so I could definitely contribute code from time to time. Seems like container.py is the part Calibre talks to and driver.py does the heavy lifting?

The plugin is great; so far I've had no troubles with annotations or bookmarks, but that's probably just as likely to be luck of the draw with my .epub structures. Also, thanks for so quickly addressing the issue with car/cbz files.

[jgoguen]

Yes, so far as I'm aware, plugins are purely Python. For me, container.py is the interface to the unpacked ePub structure, and it's where I do most of the book modification work, and driver.py does the rest of the heavy lifting. There is some modification (hyphenation mostly) outside container.py but I plan to work on moving that around later.

[Moonraker]

How do I disable the main Kobo driver when I am installing this plugin?

Calibre tells me it cannot be disabled.

Am I selecting the correct one to disable?

[bookwurm70]

Okay, so I've downloaded the kobo extended plugin. How do I use it? Will the resulting kepubs say kepub? Will I be able to use the kepub on my iPad/iPhone?

[jgoguen]

Originally Posted by Moonraker

How do I disable the main Kobo driver when I am installing this plugin?

Calibre tells me it cannot be disabled.

Am I selecting the correct one to disable?

First, make sure you don't have your Kobo device plugged in while you're doing this.

Then, in calibre open Preferences -> Plugins and look under the Device Interface Plugins list for Kobo Reader Device Interface and KoboTouch. For each one, select it and click the Enable/Disable plugin button. As long as your device isn't plugged in, they should be disabled.

[jgoguen]

Originally Posted by bookwurm70

Okay, so I've downloaded the kobo extended plugin. How do I use it?

To use the plugin, you just send books to your Kobo device.

Originally Posted by bookwurm70

Will the resulting kepubs say kepub?

On the Kobo device, yes the modified books are identified as KOBO EPUB while unmodified ePub files are identified as EPUB. If you mean in calibre, then no they aren't identified in any way because the kepub version is never saved to your calibre library.

Originally Posted by bookwurm70

Will I be able to use the kepub on my iPad/iPhone?

I don't know if you can use the generated kepubs on your iOS app. I don't see why not, but it's not something that I've tested. If you would like to try it and report back on how it worked, there may be other people who would also like to know.

[bookwurm70]

Originally Posted by jgoguen

First, make sure you don't have your Kobo device plugged in while you're doing this.

Then, in calibre open Preferences -> Plugins and look under the Device Interface Plugins list for Kobo Reader Device Interface and KoboTouch. For each one, select it and click the Enable/Disable plugin button. As long as your device isn't plugged in, they should be disabled.

Thanks! I disabled the other Kobo plugins. Now all I need is for my Aura to arrive and I'm ready to side load!

[davidfor]

Originally Posted by bookwurm70

Thanks! I disabled the other Kobo plugins. Now all I need is for my Aura to arrive and I'm ready to side load!

Sorry, you aren't ready. The drivers don't support the Aura HD yet. The next calibre release should include the changes needed.

[jgoguen]

This plugin explicitly only supports the Kobo Touch, Kobo Glo, and Kobo Mini. I don't have an Aura to test with so any updates to my plugin will have to be done by someone else with an Aura, or someone will have to send me an Aura.

EDIT: And by "explicitly only supports", I mean that this plugin will completely fail to recognize any other devices. Once the mainline Kobo driver includes Aura support, you will have to use the mainline driver and not my plugin to use calibre to send books over.

[Moonraker]

Thank you for these instructions. I have now got it to work.

I am very pleased that the problematic fonts (Nickel etc) are now working as they should and not showing italics as regular bold.

I am also pleased that my own installed fonts are showing their true weight instead of appearing anaemic and thin. The Advanced option is not available but it's no matter because my own fonts display their weight and I can still adjust font-size, line spaces and margins.

I'm not sure about the page numbers only showing number of pages left in a chapter but I can live with this because of the other advantages. I love that the title of the book is displayed in the header.

So one very happy customer here, Thank you.

[Mingyar]

Hello jgoguen,

Does the plugin work with the new kobo driver of Calibre 0.9.28 ?

What's new for 0.9.28 :
Kobo driver: Add support for the new Kobo Aura HD and firmware version 2.5.0

Thanks

[Canuck_in_Japan]

Originally Posted by jgoguen

[*]It is recommended that you disable the built-in Kobo drivers while this plugin is installed. On rare occasions, calibre may select the built-in driver instead of this plugin.

Thank you for this. I see looking at Calibre, I have Kobo Reader Device Interface and KoboTouch plugins. Should I be disabling both of those?