[Device Plugin] Kobo Touch Extended Driver

[fdanis]

Hi, recently when I upload an epub to my kobo glo there is a space, like a 0 page length chapter, between each chapters in the reading statistics.
Old converted epubs to kepubs display reading statistics correctly.
Is there a configuration option to check to fix this?

[davidfor]

Originally Posted by fdanis

Hi, recently when I upload an epub to my kobo glo there is a space, like a 0 page length chapter, between each chapters in the reading statistics.
Old converted epubs to kepubs display reading statistics correctly.
Is there a configuration option to check to fix this?

The times I have seen this is when there are two headings together with no in text between them. The structure looks like:

h2 Chapter 1
h3 In which we do stuff
Blah blah blah
h2 Chapter 2
h3 In which we do other stuff
Rhubarb rhubharb rhubharb

Sometimes the same heading level is used, sometimes different, but it is effectively a chapter with a subtitle.

If the ToC is constructed so that there is an entry for each of the heading tags above, then the h2's will be zero length chapters. But, if the ToC is built to have entries like "Chapter 1: In which we do stuff". It should be only one entry in the stats graph, but, sometimes I see two.

This is, I think, a firmware issue. The first time I saw it was the middle of last year. That would have been firmware 4.14 or 4.15. But, it could simply have been the first book I'd been reading that showed the problem. Checking that book, I can see that it is still happening with 4.20.x.

The above is roughly what I think is happening, but, I'm pretty sure there is something else involved. It would help to see another example of this. Any chance that the book is available somewhere to download legally to look at? Or, can you post a sample, or scrambled version of the book.

[fdanis]

Originally Posted by davidfor

The times I have seen this is when there are two headings together with no in text between them. The structure looks like:

h2 Chapter 1
h3 In which we do stuff
Blah blah blah
h2 Chapter 2
h3 In which we do other stuff
Rhubarb rhubharb rhubharb

Sometimes the same heading level is used, sometimes different, but it is effectively a chapter with a subtitle.

Afaict, looking at the epub file using "E-book Editor", there's only one heading level.

If the ToC is constructed so that there is an entry for each of the heading tags above, then the h2's will be zero length chapters. But, if the ToC is built to have entries like "Chapter 1: In which we do stuff". It should be only one entry in the stats graph, but, sometimes I see two.

This is, I think, a firmware issue. The first time I saw it was the middle of last year. That would have been firmware 4.14 or 4.15. But, it could simply have been the first book I'd been reading that showed the problem. Checking that book, I can see that it is still happening with 4.20.x.

The above is roughly what I think is happening, but, I'm pretty sure there is something else involved. It would help to see another example of this. Any chance that the book is available somewhere to download legally to look at? Or, can you post a sample, or scrambled version of the book.

This is reproducible with www.ebooksgratuits.com/epub/zola_germinal.epub.
I'm using version 4.19.14123

[davidfor]

Originally Posted by fdanis

Afaict, looking at the epub file using "E-book Editor", there's only one heading level.



This is reproducible with www.ebooksgratuits.com/epub/zola_germinal.epub.
I'm using version 4.19.14123

I did some testing after my post earlier, and this book confirms it, the problem is when the the ToC entries have fragments in it. With this, the graph shows the empty bar plus the proper one. Removing the fragments, which aren't needed in this case, and the graph becomes correct.

What I mean is, in the toc.ncx file, it has the following:

  <navMap>
    <navPoint id="PREMIÈRE PARTIE" playOrder="1" class="">
      <navLabel>
        <text>PREMIÈRE PARTIE</text>
      </navLabel>
      <content src="book_0001.xhtml#auto-toc0"/>
      <navPoint id="I" playOrder="2" class="">
        <navLabel>
          <text>I</text>
        </navLabel>
        <content src="book_0002.xhtml#auto-toc1"/>
      </navPoint>
      <navPoint id="II" playOrder="3" class="">
        <navLabel>
          <text>II</text>
        </navLabel>
        <content src="book_0003.xhtml#auto-toc2"/>
      </navPoint>

The fragments (marked in red) in the links are the trouble. As the target is the first tag in the each file, they aren't needed. You can edit them out manually, which is what I did to test. Or, use the ToC editor in calibre to regenerate the ToC based on major headings. Sigil should have a way to do it as well. Then remove and resend the book and it should be OK.

And, as it is a bug in the firmware, I recommend reporting it to Kobo. I will, but, it might be a couple of days before I have time to.

[thepopmonster]

After some messing around I have series info working with the new FW on a Clara HD (with the Attempt to Support Newer Firmware box checked), doesn't auto-transfer, but if I use Kobo Utilties and set "Date" as well as "Series and Index" through KoboUtilities update metadata it shows up.

However, while doing this, I was poking around in the Kobo filesystem & I noticed that I'd used two different templates in the File Formats tab and some books in a series are in different folders than others (I use {series:| - |} as part of the folder name now but apparently used to use /{series}/).

Now, I could delete the affected books and then re-add them, but that's both tedious and manual and messes up the relationship between purchased order and added order.

So, I guess I have 3 questions:

1. Is there some way that I can get Calibre to move all books already on the device to match my template?
2. Should I be asking this in KoboUtiltities instead?
3. Assuming I do have to wipe and reload, is there something I can do during that process to make Kobo Date = Calibre date added, or at least in the same relative order, if I am sending all books in the one operation (there are more distinct dates and books than I want to do manually), or is this another question for KoboUtilities?

Thanks in advance.

Device database version: 158.
Device firmware version: (4, 20, 14601)

[davidfor]

Originally Posted by thepopmonster

After some messing around I have series info working with the new FW on a Clara HD (with the Attempt to Support Newer Firmware box checked), doesn't auto-transfer, but if I use Kobo Utilties and set "Date" as well as "Series and Index" through KoboUtilities update metadata it shows up.

The driver definitely should be updating the series info if the option is turned on. Don't forget that for new books, it is done on the next connection. And the Kobo Utilities plugin cannot do anything until the book has been imported on the device.

But, the most common reason for the metadata not being updated automatically is that the "Metadata managment" option on the "Sending books to devices" page of the preferences is not set to automatic. If it is set to something else, the metadata updates and collection management are not done until a book is sent to the device.

However, while doing this, I was poking around in the Kobo filesystem & I noticed that I'd used two different templates in the File Formats tab and some books in a series are in different folders than others (I use {series:| - |} as part of the folder name now but apparently used to use /{series}/).

Now, I could delete the affected books and then re-add them, but that's both tedious and manual and messes up the relationship between purchased order and added order.

So, I guess I have 3 questions:

1. Is there some way that I can get Calibre to move all books already on the device to match my template?
2. Should I be asking this in KoboUtiltities instead?
3. Assuming I do have to wipe and reload, is there something I can do during that process to make Kobo Date = Calibre date added, or at least in the same relative order, if I am sending all books in the one operation (there are more distinct dates and books than I want to do manually), or is this another question for KoboUtilities?

I have considered what to do when changing the save template, or changing something in the metadata that used in the save template. I haven't found any good way to handle this automatically. If it is done, it means a lot of changes to the database, and there is always a risk there.

What I have done in the past when I have done a mass update like this is to use the Kobo Utilities plugin to set the "Date Added" on the device. The plugin can use the "Date", "Modified" or publishing date from the calibre library. I usually use the "Date" as that defaults to the date the book was added to the library. I don't find that "Modified" works very well as there are some things that can trigger a change to this for all books (add a new column) or a lot of books (rename a tag).

[thepopmonster]

Originally Posted by davidfor

The driver definitely should be updating the series info if the option is turned on. Don't forget that for new books, it is done on the next connection. And the Kobo Utilities plugin cannot do anything until the book has been imported on the device.

But, the most common reason for the metadata not being updated automatically is that the "Metadata managment" option on the "Sending books to devices" page of the preferences is not set to automatic.

Thanks, that was exactly the problem - metadata was set to "only on send" in the Calibre preferences (probably from a while back when I didn't have a template set at all and it took minutes whenever I connected my android devices).

Looks like I need to set aside some time to reload.

For general reference, this seems to work fairly well across both android and Kobo if you remove periods/ellipses from the series names (e.g. March... -> March to the Stars)

{author_sort:.1}/{author_sort}{series:| - |}/{series_index:0>5.2f} {title}

[jgoguen]

One more call for anyone who wants to test the new plugin version before it's released, get the updated files for manual install from https://www.mobileread.com/forums/sh...postcount=1973

Included in this update is Python 3 compatibility and logging updates to hopefully make debugging easier. I don't currently have a working Kobo device so advance testers are important!

[Terisa de morgan]

I'm finding this error with the beta driver:

Spoiler:

'utf8' codec can't decode byte 0xe1 in position 28: invalid continuation byte

Traceback (most recent call last):
File "site-packages\calibre\gui2\device.py", line 90, in run
File "site-packages\calibre\gui2\device.py", line 513, in _books
File "site-packages\calibre\devices\kobo\driver.py", line 1599, in books
File "calibre_plugins.kobotouch_extended.device.driver" , line 113, in settings
File "calibre_plugins.kobotouch_extended.common", line 90, in print_formatted_log
File "calibre_plugins.kobotouch_extended.common", line 79, in _tag_args
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe1 in position 28: invalid continuation byte

Edit: I con confirm that the problem is the version, as it works OK with the official version.

[jackie_w]

Originally Posted by jgoguen

One more call for anyone who wants to test the new plugin version before it's released, get the updated files for manual install from https://www.mobileread.com/forums/sh...postcount=1973

Can I make a suggestion that before you release your new version of KTE that you correct the included hyphenation.css file.

The existing file has always contained invalid webkit CSS properties:

-webkit-hyphenate-after: 3;
-webkit-hyphenate-before: 3;
-webkit-hyphenate-lines: 2;

For kepubs those property names should be:

-webkit-hyphenate-limit-after: 3;
-webkit-hyphenate-limit-before: 3;
-webkit-hyphenate-limit-lines: 2;

I don't use the last one, -webkit-hyphenate-limit-lines, myself but the first 2 seem to respond OK when you change the values.

[JSWolf]

Originally Posted by jackie_w

Can I make a suggestion that before you release your new version of KTE that you correct the included hyphenation.css file.

The existing file has always contained invalid webkit CSS properties:

-webkit-hyphenate-after: 3;
-webkit-hyphenate-before: 3;
-webkit-hyphenate-lines: 2;

For kepubs those property names should be:

-webkit-hyphenate-limit-after: 3;
-webkit-hyphenate-limit-before: 3;
-webkit-hyphenate-limit-lines: 2;

I don't use the last one, -webkit-hyphenate-limit-lines, myself but the first 2 seem to respond OK when you change the values.

Can you please report this for kepubify as well? This is the code that kepubify adds in to each HTML. Thanks.

    -webkit-hyphenate-after: 3;
    -webkit-hyphenate-before: 3;
    -webkit-hyphenate-lines: 2;
    hyphenate-after: 3;
    hyphenate-before: 3;
    hyphenate-lines: 2;

[jackie_w]

Originally Posted by JSWolf

Can you please report this for kepubify as well? This is the code that kepubify adds in to each HTML. Thanks.

Well, it wouldn't be right for me to report problems in utilities I've never personally used, e.g. Kepubify.

If you do use Kepubify yourself, feel free to report it. You can always link to my KTE post to save yourself some typing.

Hint: Make absolutely sure that Kepubify is currently using the same faulty CSS if you don't want a flea in your ear from geek1011

[JSWolf]

Originally Posted by jackie_w

Well, it wouldn't be right for me to report problems in utilities I've never personally used, e.g. Kepubify.

If you do use Kepubify yourself, feel free to report it. You can always link to my KTE post to save yourself some typing.

Hint: Make absolutely sure that Kepubify is currently using the same faulty CSS if you don't want a flea in your ear from geek1011

Yes it is using the same. What I posted if from the latest kepubify.

[Svens]

Error with the 3.2.2 KoboTouchExtended. Calibre ver. 4.13, Kobo Aura One /fw 4.20.14622.
With ver. 3.1.5 all is OK, but series not show on Kobo.
Only seriesmeta-windows-64bit.exe works fine.

'utf8' codec can't decode byte 0xd4 in position 20: invalid continuation byte

Traceback (most recent call last):
File "site-packages\calibre\gui2\device.py", line 90, in run
File "site-packages\calibre\gui2\device.py", line 513, in _books
File "site-packages\calibre\devices\kobo\driver.py", line 1565, in books
File "site-packages\calibre\devices\kobo\driver.py", line 3542, in get_debugging_title
File "site-packages\calibre\devices\kobo\driver.py", line 3264, in get_pref
File "calibre_plugins.kobotouch_extended.device.driver" , line 113, in settings
File "calibre_plugins.kobotouch_extended.common", line 90, in print_formatted_log
File "calibre_plugins.kobotouch_extended.common", line 79, in _tag_args
UnicodeDecodeError: 'utf8' codec can't decode byte 0xd4 in position 20: invalid continuation byte

[davidfor]

Originally Posted by Svens

Error with the 3.2.2 KoboTouchExtended. Calibre ver. 4.13, Kobo Aura One /fw 4.20.14622.
With ver. 3.1.5 all is OK, but series not show on Kobo.
Only seriesmeta-windows-64bit.exe works fine.

'utf8' codec can't decode byte 0xd4 in position 20: invalid continuation byte

Traceback (most recent call last):
File "site-packages\calibre\gui2\device.py", line 90, in run
File "site-packages\calibre\gui2\device.py", line 513, in _books
File "site-packages\calibre\devices\kobo\driver.py", line 1565, in books
File "site-packages\calibre\devices\kobo\driver.py", line 3542, in get_debugging_title
File "site-packages\calibre\devices\kobo\driver.py", line 3264, in get_pref
File "calibre_plugins.kobotouch_extended.device.driver" , line 113, in settings
File "calibre_plugins.kobotouch_extended.common", line 90, in print_formatted_log
File "calibre_plugins.kobotouch_extended.common", line 79, in _tag_args
UnicodeDecodeError: 'utf8' codec can't decode byte 0xd4 in position 20: invalid continuation byte

That looks to be an error in a debugging statement to write the settings for the driver to the log. Can you post a copy of the configuration file? This the file "device_drivers_KOBOTOUCHEXTENDED.py.json" and is in the calibre configuration directory. I think the most likely issue is that there is a character in the option "Copy generated KePub files to directory" that isn't being handed properly. Or the "Title to test when debugging" field. Clearing these will probably fix it, but, seeing the config file will let us check.


And the fact that seriesmeta-windows-64bit.exe works has absolutely nothing to do with anything here. There is no shared code and they act on different data sources. If they both failed at the some time, it would be a coincidence. Or mean that you would probably be doing a factory reset of your Kobo device in the near future.