[Device Plugin] Kobo Touch Extended Driver

[Leonatus]

Originally Posted by jgoguen

If the file you have is a proper TAR archive, you can just GZip it and name the resulting file KoboRoot.tgz. If you created a gzipped TAR archive and just named it wrong, then you should be able to change the extension.

Yes, yes, yes. I have been unattentive. I had created the archive, but copied the wrong file into the .Kobo folder. Luckily, there happened nothing. Now I have obviously done it correctly, sideloaded a new book via calibre, and as far as I see, almost all hyphens are set properly.

It's a charm!

[BobC]

Tried to run the conversion plugin and I get the following error :

Spoiler:

calibre, version 0.9.42 (win32, isfrozen: True)
Conversion Error: Failed: Convert book 1 of 1 (Test nbsp1)

Convert book 1 of 1 (Test nbsp1)
Resolved conversion options
calibre version: 0.9.42
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0.0,
'book_producer': None,
'change_justification': u'original',
'chapter': u"//*[name()='h1' or name()='h2' or name()='h3'] ",
'chapter_mark': u'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_split_on_page_breaks': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'epub_flatten': False,
'epub_inline_toc': False,
'epub_toc_at_end': False,
'extra_css': None,
'extract_to': None,
'filter_css': u'',
'fix_indents': True,
'flow_size': 260,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': u'utf-8',
'input_profile': <calibre.customize.profiles.InputProfile object at 0x03D224D0>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0.0,
'linearize_tables': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'no_chapters_in_toc': False,
'no_default_epub_cover': True,
'no_inline_navbars': False,
'no_svg_cover': False,
'opt_kepub_clean_markup': True,
'opt_kepub_enable_extended_features': True,
'opt_kepub_hyphenate': True,
'opt_kepub_replace_lang': True,
'output_profile': <calibre.customize.profiles.KoboReaderOutput object at 0x03D22890>,
'page_breaks_before': u'/',
'prefer_metadata_cover': False,
'preserve_cover_aspect_ratio': False,
'pretty_print': True,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': u'C:\\DOCUME~1\\BobC\\LOCALS~1\\Temp\\calibre_ycad hc\\obcs3k.opf',
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': True,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': u'',
'search_replace': '[]',
'series': None,
'series_index': None,
'smarten_punctuation': False,
'sr1_replace': None,
'sr1_search': None,
'sr2_replace': None,
'sr2_search': None,
'sr3_replace': None,
'sr3_search': None,
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': True,
'verbose': 2}
InputFormatPlugin: ODT Input running
on C:\DOCUME~1\BobC\LOCALS~1\Temp\calibre_ycadhc\_vw5 cw.odt
Extracting ODT file...
Parsing all content...
Parsing index.xhtml ...
Parsing odfpy.css ...
Generating default TOC from spine...
Merging user specified metadata...
Detecting structure...
Auto generated TOC with 0 entries.
Flattening CSS and remapping font sizes...
Source base font size is 12.00000pt
Removing fake margins...
Found 7 items of level: p_1
Ignoring level p_1
Cleaning up manifest...
Trimming unused files from manifest...
Creating KePub Output...
Splitting markup on page breaks and flow limits, if any...
Looking for large trees in index.xhtml...
No large trees found
This EPUB file has no Table of Contents. Creating a default TOC
Python function terminated unexpectedly
[Errno 2] No such file or directory: '/home/jgoguen/output.opf' (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 189, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 31, in gui_convert_override
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 1177, in run
File "calibre_plugins.kobotouch_extended.conversion.kep ub_output", line 69, in convert
File "shutil.py", line 83, in copyfile
IOError: [Errno 2] No such file or directory: '/home/jgoguen/output.opf'

This is a very simple "book" but I get the same failure on a few others.

BobC

[jgoguen]

Here's new files, they should be wholly independent. The Git repo has also been updated.

[BobC]

Originally Posted by jgoguen

Here's new files, they should be wholly independent. The Git repo has also been updated.

That solved both my problems - I've used the attached copies rather than building them myself this time - though I now have both the Conversion plugin and the extended driver installed so I can't confirm that they are fully independent.

Minor (non-functional) issues with the converter :

In the Kepub Output options of the conversion dialogue there are no Tool Tips for the extra options that go with Kepub. It is for instance less than clear what "Enable Kepub Extended Features" does; in the driver I thought it produced Kepub files with all the necessary kobo spans etc. i.e it produced Kepubs , if that is so then the option seems unnecessary in a converter.

I also did not tick the Hyphenate option but it looks like the kepub produced by the converter included kte-css/hyphenate.css and a link to it from each of the xhtml files. Is this intended ?

HTH BobC

[jgoguen]

I see your point about the extended features option. Perhaps that can get taken out

I haven't found a way to get the tooltips yet. There's probably something simple I'm missing, I'll be going through some of the built-in plugins to see what's missing.

Hyphenation CSS should only be there if you select hyphenation.

[jgoguen]

New versions up for testing.

[BobC]

Originally Posted by jgoguen

New versions up for testing.

That seems to have fixed the issues I mentioned. I haven't checked all the bookmarking, highlighting etc facilities (I don't use them much).

Thank you.

BobC

[anacreon]

I've just realized I posted this in the wrong thread, and since I don't know how to move it, I copy it, modified, hereunder.

I decided recently to use the kobotouch extended plugin for my annotated epubs, because it is the only convenient way to read notes. Also for very big "omnibuses", such as the Pynch editions, I like to know where I am, so the kepub footer is the best answer.

But I have a problem with Addison and Steele's Spectator, found on Project Gutenberg: with the plugin activated, Calibre silently sends a plain epub to my aura. At 2.2 MB and 2208 pages this epub's size isn't that different from another epub at 2.8 MB (before conversion => 3.4 MB after conversion) and 2109 pages which works fine.

Conversely, I had problems with Pynch's Kafka's Gesammelte Werke in the kepub version (everything in italics), so I cancelled it from the aura, deactivated the kobo extended plugin, and sent it back to the device, with others collected works, and they all are kepubs. I've had the problem occasionnaly, but usually desactivating works. So I had to uninstall the plugin before the transfer.

NB: I have the latest versions of everything (kobo, calibre, plugins).

[BobC]

Originally Posted by anacreon

But I have a problem with Addison and Steele's Spectator, found on Project Gutenberg: with the plugin activated, Calibre silently sends a plain epub to my aura. At 2.2 MB and 2208 pages this epub's size isn't that different from another epub at 2.8 MB (before conversion => 3.4 MB after conversion) and 2109 pages which works fine.

An observation - are you just using the PG Epub "as-is" ? In general I find that they need a lot of tidying up to go cleanly onto a Kobo. The main issue is that PG splits the files simply according to length whereas for a decent reading experience you need to split them on a chapter basis.

My usual approach is to use Sigil to merge all the contents and then set markers based on chapters and then split the files based on the markers. You then might need to tidy up the TOC and clean up any other errors.

It makes for quicker opening of the chapters and always get a new chapter on a new page not halfway down a page and chapters split across pages apparently randomly.

I am now using jgoguen's KePub conversion plug-in to do the conversion before transferring, that way I should be able to see if there is likely to be an issue that will make the conversion fail (and perhaps just upload the EPUB).

BobC

[jgoguen]

Originally Posted by anacreon

with the plugin activated, Calibre silently sends a plain epub to my aura.

The default behaviour is to throw an exception, there is an option to silently ignore conversion errors but I generally recommend not setting it. You can also get a plain ePub file if you have unselected the option to use the extended features, or if you haven't disabled the KoboTouch driver and calibre loads that driver instead.

If you have not selected the option to silently ignore errors and you have enabled extra features, please see the first post in this thread for how to submit a bug report. Considering that you're using a Project Gutenberg ePub, you may first want to consider asking around the Sigil forum (or perhaps BobC has some pointers as well) for help with manipulating the book. Like BobC mentioned, there may be issues with the books that cause issues with KePub viewing or even KePub generation. BobC is exactly right on chapter splitting, the KePub viewer assumes one chapter per file (no matter how long the chapter is).

Originally Posted by anacreon

So I had to uninstall the plugin before the transfer.

You shouldn't have needed to do anything more than uncheck the "extended features" option. Then you get everything else you've selected except not the special Kobo features and you get a plain ePub file and not a KePub. If that didn't work, please submit a bug report as detailed in the first post of this tread.

[BobC]

Another good reason for tidying up PG Epubs as I have just discovered, they may contain .png images which are valid in Epubs but the current version of Kobo s/w seems to have problems with them. I've just had to convert all the .png images to .jpg in a couple of books.

BobC

[anacreon]

Originally Posted by jgoguen

The default behaviour is to throw an exception, there is an option to silently ignore conversion errors but I generally recommend not setting it. You can also get a plain ePub file if you have unselected the option to use the extended features, or if you haven't disabled the KoboTouch driver and calibre loads that driver instead.

If you have not selected the option to silently ignore errors and you have enabled extra features, please see the first post in this thread for how to submit a bug report. Considering that you're using a Project Gutenberg ePub, you may first want to consider asking around the Sigil forum (or perhaps BobC has some pointers as well) for help with manipulating the book. Like BobC mentioned, there may be issues with the books that cause issues with KePub viewing or even KePub generation. BobC is exactly right on chapter splitting, the KePub viewer assumes one chapter per file (no matter how long the chapter is).

Thank you, and thanks to BobC. I had indeed selected the "silently ignore errors", and I've corrected that. I started working on the Spectator book, but since the paper was a daily, with 635 numbers, I gave up. I'll just read the plain epub.

Originally Posted by jgoguen

You shouldn't have needed to do anything more than uncheck the "extended features" option. Then you get everything else you've selected except not the special Kobo features and you get a plain ePub file and not a KePub. If that didn't work, please submit a bug report as detailed in the first post of this tread.

Testing, I've seen that I have to restart calibre after disactivating the plugin for it to work. So now I have plain epubs Spectator and Kafka on my aura. Pity, I like your plugin - I still don't understand why the Kafka switches to all italics when turned into a kepub.

[BobC]

I have just completed editing an EPUB to get the chapters etc right and then converted it to KEPUB. I then Exploded and Examined the Kepub and came across some strange results.

I had split the epub into Chapters which started with a h2 header with the text "CHAPTER n" followed by another h2 header with a subtitle text e.g. "Our Hero Goes Forth".

When I convert this to kepub the chapters are split so section_004.html becomes section_004_split_001.html and section_004_split_002.html with the first file containing just the CHAPTER n heading and the second one the rest of the original. Needless to say this means that when rendered on the Glo the combining I had done is undone and CHAPTER n will appear on a separate page.

I checked that I have nothing in the "Detect Chapters" xpath under "Stucture Detection" in the conversion options.

From the conversion log it appears that this is where the splitting is happening :

Splitting markup on page breaks and flow limits, if any...
Splitting on page-break at id=calibre_pb_18
Splitting on page-break at id=calibre_pb_19
Splitting on page-break at id=calibre_pb_20

This is the code in the original html which is triggering the split :

<h2 class="calibre23" id="calibre_pb_18"><span class="calibre18"><span class="bold"><span><a class="calibre15" href="../Text/Nation_002.html#filepos1740"><span class="calibre16"><span><span class="bold">CHAPTER 3</span></span></span></a></span></span></span></h2>

I'm unclear as to why these are being treated as pagebreaks - AIUI the id is just reference.

Is this a bug that needs reporting or have I missed some obscure option ?

BobC

EDIT: I've looked at another conversion and it looks like the page split is happening on all id= ocurrences

EDIT2 : Found the option for "Do Not Split on Page Breaks" - if selected this stops the unwanted splitting, however why all id's are being treated as page breaks is less than obvious. I suspect this is not a function of the Kepub conversion but of Calibre's EPUB routines.

[jgoguen]

Originally Posted by anacreon

Testing, I've seen that I have to restart calibre after disactivating the plugin for it to work. So now I have plain epubs Spectator and Kafka on my aura. Pity, I like your plugin - I still don't understand why the Kafka switches to all italics when turned into a kepub.

If you want to use my plugin but have the option to send ePub files for some things, enable my plugin and disable KoboTouch, restart calibre, then in my plugin options enable the extended features to get a KePub and disable extended features to get an ePub with the other options you choose (hyphenation, cleanup, etc.). If you go to calibre preferences, Plugins, then the plugin options, you don't need to restart calibre after changing options.

[anacreon]

Originally Posted by jgoguen

If you want to use my plugin but have the option to send ePub files for some things, enable my plugin and disable KoboTouch, restart calibre, then in my plugin options enable the extended features to get a KePub and disable extended features to get an ePub with the other options you choose (hyphenation, cleanup, etc.). If you go to calibre preferences, Plugins, then the plugin options, you don't need to restart calibre after changing options.

Thank you, it is indeed quicker than restarting calibre, since I have 3500+ books in calibre, 1800+ of which in the virtual library aura, which means the time needed to connect / disconnect reader to calibre is way longer than with the 1000+ glo.