Kobo Device Driver Update

[davidfor]

Originally Posted by jackie_w

My SaveTemplate does include Series info

Books/{author_sort} - {series}{series_index:0>2s| | - }{title}

I also have a metadata plugboard remapping Author in case that's relevant.

OK, but I'll remove the save template/plugboard variables first and see how that goes.

The plugboard doesn't affect this. If it is what I think it is, then it is related to the save template. And it usually only happens to books that are on the device that calibre doesn't know about.

Neither of the 'no longer detected' books have multiple authors. Only one of my current books on the Glo has multiple authors but it has no series and does have the 'series set to first char of title' problem.

Ok, the multiple authors can break the matching sometimes. Can you send me the metadata.calibre file from the device? Plus the name of some of the books with problems.

That would be nice to have but don't put it high on your to-do list on my account.

I have a couple of little bugs to fix, then I'll do this. Or maybe I'll read a book

Here is some extra info:
All the books are standard epub with no DRM.

In case my problems are self-inflicted perhaps this is also relevant. Whilst the normal workflow would be:

1. Connect and send-to-disk
2. Disconnect and let reader process new books.
3. Repeat connect/disconnect

I have been adding a semi-automated process (not via calibre code) between 1 and 2. For each newly-sent book on the Kobo disk

- open epub
- append kobo-specific css to existing css file
- rebuild epub in situ

Both file date/time and filesize will therefore increase slightly. Understanding the logic of how things work, can you see why this might cause problems? I would add that I have been doing this for months on fw v2.1.5 with no apparent problems.

None of that should cause any problems. I do this occasionally when testing things. As long as the epub is still valid, it should be loaded OK.

I also tried adding my extra process after connecting for a 3rd time, but on disconnect the Glo deleted all updated epubs. This method also used to work OK on v2.1.5

This will cause problems. Across the firmware versions, the books have been checked for changes. And sometimes deleted. What was checked has changed. With recent firmware, it is the file size. If the file size does not match what is in the database, all evidence of the book will be removed. I put a change in the driver a while ago that updates the size when a book is resent to the device. Your changes after sending the book will probably change the file size. You can fix that by updating the database.

I do something like this for some books I download from a couple of sites. But, I do it in the library. I hacked the Modify ePub plugin to replace the CSS file plus do a couple of other tweaks. Otherwise, I just edit them in Sigil or using Tweak books. If you have any Python skills, have a look at the extended plugin. It makes changes after the epub is copied. And because it subclasses the main driver, it inherits all the other function. It wouldn't be hard to take this and change it to replacing something else in the epub.

I don't expect the Kobo driver to be changed to accommodate my niche requirements but some understanding of problems I may be creating for myself would be appreciated. I can probably find a workaround if I have to.

No, but when you discuss things like this, you frequently get others saying they do it as well. And you might give me an idea.

[Bittybye]

Originally Posted by davidfor

Is this when you send the book to the AuraHD? How soon after starting the send? What happens when you copy a book to the device without calibre?

I'm assuming you were talking to me, almost instantly after I send the book to the Aura. And I haven't tired directly adding a book. But it happens when I add to the Kobo itself and the SD card.

[GeoffR]

Originally Posted by davidfor

What I can do is to change the "Set series info" option to "Set other metadata". Then have it set the series info, publisher, published date and ISBN. I already have code to do most of it. The only concern is performance. The first connect after sending a lot of books will take more time, but after that, it should be OK.

Opinions anyone?

This would be a nice feature, I know the publisher/date/ISBN info is usually in the book but it is good to be able to see it without opening the book and the various changes to reading status etc. that entails.

I think those people who like to manage the series information for books on the device would probably be the ones who would also like to see the other metadata, and so might be happy with it taking a bit extra time after sending new books.

The only issue I see is that those who want to manage the series info by appending it to the book title and/or creating series shelves instead of using Kobo's series fields would not be able to add just the publisher/date/ISBN info. But then they would be no worse off than they are now without the option.

[jackie_w]

@davidfor,

After experimentation with mixed results, this is my update on

Problem 1: For books without Series data, 7 of 8 had their Kobo Series field populated with the first char of the book Title after the follow-up connect/disconnect cycle.

Problem 2: For books with Series data, 2 of 24 books did not register a tick in the 'On device' column after the follow-up connect, even though they were still on the Kobo. Hence the Series data never got transferred and the Kobo series display was blank.

Re: problem 1: I simplified the KoboTouch save template from

Books/{author_sort} - {series}{series_index:0>2s| | - }{title}

to

Books/{author_sort} - {title}

I'm pleased to say this seems to have fixed it. As I'm not desperately attached to my original Save Template, I'll consider this problem finished.

Re: problem 2: Changing the Save Template didn't fix this one, but simplifying the metadata plugboard to

{author_sort} --> authors

from a combination of author_sort and abbreviated series (more detail in the attached zip) did make the problem go away.

I guess this will be an acceptable workaround for now, but I set up the original plugboard so that Kobo booklists, when sorted by Author, were automatically in Author/Series/index sequence. If I can get that back I'll be even happier. I used the latter plugboard for the last 6 months without problems on fw 2.1.5

The debug logs and metadata.calibre copies for problem 2 are in the attached zip, with a step-by-step of exactly what I did detailed in the ReadMe file.

Good luck, and thanks for your help so far

[jackie_w]

@david,

Additional info. It looks like my above remaining problem with fw2.5.1 is even more niche than I thought.

If I have both of these conditions then I get some books not recognised by calibre after the 2nd connect:

- a plugboard containing series info
- epub tweaking between initial download and calibre disconnect

However, if I have one but not the other then there is no apparent problem (I didn't try with enough books to be 100% certain). My guess is that most people won't be doing both of the above. Do you have any insight into why these 2 conditions combined might create the 'undetected' problem? I would have thought that book matching used the book id rather than author/title metadata, but maybe not.

If I want both, I'm going to have to find a way to tweak the epubs before send-to-device. As you suggested, the neatest way would be for me to try something similar to jgoguen's device plugin clone. Whilst I have limited (very) Python skills that get me by, trying to unravel the depth and mysteries of calibre classes has never ended well before (akin to being in the middle of a labyrinth with no string attached ) Maybe I'll pluck up the courage to have another go.

I suppose I could also clone my on-device books to a separate calibre mini-library, do the tweaks there, then send-to-device. Easy enough to do initially, but an unwanted, dreary maintenance chore.

Finally, I tried all the earlier tests on my new Aura, with pristine, unsullied database. The results were all the same as before, so I'm not going to bother doing a factory reset of the Glo to see if that makes a difference.

Edit: Managed to cobble together a workaround. Not as elegant as a bespoke device driver but it'll do.

[PeterT]

David; is there any way of warning a user if the Automatic Metadata management option is not set when they use the new driver (or maybe have shelving options defined)?

[davidfor]

Originally Posted by PeterT

David; is there any way of warning a user if the Automatic Metadata management option is not set when they use the new driver (or maybe have shelving options defined)?

Other than when the driver configuration is closed, I'm not sure when to display it. Doing at it at that time when the shelf or series options are set would be a good idea.

[davidfor]

With calibre 0.9.42, there is a new function added to the KoboTouch driver. This is to modify the stylesheets in an ePub during the send. The idea is to add CSS rules to the existing stylesheets in the ePubs to change the way a book is displayed.

Setup
To use this, you need to do the following:

• Check the new option "Modify CSS" on the driver options. Note: if you have previously had the "Attempt to support new firmware" option selected, this might already be selected.
• Create a file called "kobo_extra.css" in the root directory of your Kobo eReader
• In this file, add any CSS rules that you want for all your books.
• Send the books to your Kobo

What Happens
Then, when sending the books, the driver reads "kobo_extra.css" and does the following:

• If there are any "@page" rules in "kobo_extra.css", it removes any existing @page rules from the stylesheets in the epub.
• If any rules in "kobo_extra.css" have "widows" or "orphans" setting, all "widows" or "orphans" are removed from CSS rules in the stylesheets.
• The contents of "kobo_extra.css" are appended at the end of all stylesheets in the ePub.

Example CSS Rule
A simple example of how this can be used is to put the following in "kobo_extra.css":

body {
	orphans: 1;
	widows: 1;
	}

That will allow the text to more fully fill the screen rather than keep lines in a paragraph together.

Disclaimer and Warnings

• Use this at your own risk. These changes have been tested extensively, but, there still could be problems.
• If a book is protected by DRM, it cannot be modified. It will be sent to the device unchanged.
• If a book is already on the device, it must be resent to be modified.
• The contents of "kobo_extra.css" must be valid CSS rules. If they are not, the ePub will probably not be displayed on the device properly.
• The instructions above probably don't make sense. I'll fix them as people ask questions. Or if I'm really lucky, someone who can string sensible sentences together will do it for me

Acknowledgements
This would not have happened without help from people around here. jackie_w supplied a prototype for the changes which worked very well, and then tested and critiqued my changes. jgoguen helped with making sure it was compatible with the KoboTouchExtended driver. And supplied some much needed suggestions and testing. As always, Kovid of course was there when needed. Plus my beta testers who I always pick on

Support
As always, if you find a problem with the driver, please report it here. In detail. The more detail, the better. In this case, I will probably need to see the ePub that you are having problems with and the contents of your "kobo_extra.css". If the ePub is under copyright, please do not post it here. I will arrange another way to get it.

[buffaloseven]

Using the kobo_extra.css to fix widows/orphans on my Kobo Glo and it's working wonderfully. Thanks for the update!

[speakingtohe]

Originally Posted by davidfor

Then, when sending the books, the driver reads "kobo_extra.css" and does the following:

• If there are any "@page" rules in "kobo_extra.css", it removes any existing @page rules from the stylesheets in the epub.
• If any rules in "kobo_extra.css" have "widows" or "orphans" setting, all "widows" or "orphans" are removed from CSS rules in the stylesheets.
• The contents of "kobo_extra.css" are appended at the end of all stylesheets in the ePub.

Example CSS Rule
A simple example of how this can be used is to put the following in "kobo_extra.css":

body {
	orphans: 0;
	widows: 0;
	}

I followed your instructions and the CSS is appended as stated.
Could you tell me how I would go about modifying line height in a typical calibre converted epub?

I put line-height: 1.0; in the kobo_extra.css file and it is there but no effect on the epub. The line-height is appended, but not removed elsewhere.
Probably I am misunderstanding something simple.

Helen

Helen

[jackie_w]

Originally Posted by speakingtohe

Could you tell me how I would go about modifying line height in a typical calibre converted epub?

I put line-height: 1.0; in the kobo_extra.css file and it is there but no effect on the epub. The line-height is appended, but not removed elsewhere.

The plugin doesn't remove all existing occurrences of line-height. It only does that for widows & orphans and @page.

If you know that your epubs have all been calibre-converted at some point then you may be able to use kobo_extra.css to override the Kobo's own line-height setting on the font adjustment screen, but it will depend on how much each epub is riddled with other line-height settings.

Try adding something like

body.calibre, body.calibre1, body.calibre2, body.calibre3, body.calibre4, body.book, body.epub {
    line-height: 1.0
    }

to your kobo_extra.css file. If that isn't successful enough you could try adding div, p to the selector list, i.e.

body.calibre, body.calibre1, body.calibre2, body.calibre3, body.calibre4, body.book, body.epub, div, p {
    line-height: 1.0
    }

and see if that 'catches' more of your epubs. But it may also change line-height settings you wish it hadn't.

Just be aware that if this does work for any given epub, it is likely that the Kobo's own line spacing adjustment bar will no longer work for that epub -- but you may not care about that.

[speakingtohe]

Originally Posted by jackie_w

The plugin doesn't remove all existing occurrences of line-height. It only does that for widows & orphans and @page.

If you know that your epubs have all been calibre-converted at some point then you may be able to use kobo_extra.css to override the Kobo's own line-height setting on the font adjustment screen, but it will depend on how much each epub is riddled with other line-height settings.

Try adding something like

body.calibre, body.calibre1, body.calibre2, body.calibre3, body.calibre4, body.book, body.epub {
    line-height: 1.0
    }

to your kobo_extra.css file. If that isn't successful enough you could try adding div, p to the selector list, i.e.

body.calibre, body.calibre1, body.calibre2, body.calibre3, body.calibre4, body.book, body.epub, div, p {
    line-height: 1.0
    }

and see if that 'catches' more of your epubs. But it may also change line-height settings you wish it hadn't.

Just be aware that if this does work for any given epub, it is likely that the Kobo's own line spacing adjustment bar will no longer work for that epub -- but you may not care about that.

Thanks, will give it a try.
And no I don't really care as I actually want a smaller line height
Helen

PS
Didn't work. Didn't even make it into the CSS file, perhaps this is just for widows and orphans which I already have in my conversion defaults.

I must try and learn more about CSS and the overriding of

Thanks again

[davidfor]

Originally Posted by speakingtohe

Thanks, will give it a try.
And no I don't really care as I actually want a smaller line height
Helen

PS
Didn't work. Didn't even make it into the CSS file, perhaps this is just for widows and orphans which I already have in my conversion defaults.

I must try and learn more about CSS and the overriding of

Any valid CSS in the file should be added to the stylesheets. Can you post what you have tried?

[speakingtohe]

Originally Posted by davidfor

Any valid CSS in the file should be added to the stylesheets. Can you post what you have tried?

Widows and orphans worked.
Line-height, font size were posted but had no effect.

From jackie_w
The Code:
body.calibre, body.calibre1, body.calibre2, body.calibre3, body.calibre4, body.book, body.epub {
line-height: 1.0
}

This did not go into the file.
I thought it might be a bit long so I shortened it in various ways but no luck.
Put it in the book directly using tweak epub with no change in appearance (I used different line heights to make it more visible. But when I put it in the epub directly the code was visible in the calibre viewer web inspector I believe just had
lines through it.

When I put line-height in the kobo css file it was visible in the inspector, but again crossed out.

I flailed around for a bit and got quite confused, so decided to leave it alone for a while.

Helen

[davidfor]

Using the CSS that jackie_w posted, was added to the stylesheet OK. But, it didn't do what you wanted.

I think the problem is the CSS rules. A calibre conversion creates style rules like:

.calibre2 {
    color: maroon;
    display: block;
    font-size: 2em;
    font-weight: 100;
    line-height: 200%;
    margin-bottom: 0.67em;
    margin-left: 0;
    margin-right: 0;
    margin-top: 3em;
    text-align: center
    }

Each of these are used with a tag. And each is used with only one tag. Exactly which tag the number style is used with depends on the formatting of the book. As some of the styles are used with the body, some with a paragraph and some with other tags, using "body.calibre2" will not work if "calibre2" is only used with a paragraph. Or at least, that is my understanding of CSS. Inspecting the epub like you have and seeing the style crossed out, supports this.

Try:

body, p, .calibre, .calibre1, .calibre2, .calibre3, .calibre4, body.book, body.epub {
line-height: 1.0
}

The only question is how many ".calibreXX" styles to put in. I have seen over 30 after a conversion.

Its possible that "p.calibre1" should be used. That should mean overriding the line-height only when the .calibreXX is used in a paragraph tag.