Kepub format questions

[jackie_w]

Originally Posted by geek1011

Opinions are what creates discussion. ... Thanks for the feedback! This will probably clear up other people's questions as well.

This is where my 'can of worms' sensor went off...

and some of the built-in CSS mods (such as the full screen fixes until recently, the CSS inliner to deal with the buggy FW and some buggy books, etc)

... seems to allow for auto-incorporating what would normally be applied via a patch.

- Apply CSS modifications to fix or tweak bugs (see above) or Kepub features (like hyphenation)

... seems to be a prime candidate for conflicting with the 'Modify CSS' option in the KT/KTE drivers and the Hyphenation option in KTE.

- Apply fixes to book-breaking (i.e. book is not readable) CSS and HTML layout which are in multiple books by a publisher

If there are a well-defined set of problems with a source book why fix the kepub? Why not fix the source instead? That's what the Modify Epub (ME) and Quality Check (QC) plugins were supposed to be for.

- Remove extra MS/ADE/iBooks/Calibre cruft

... again, if there is MS/ADE/iBooks cruft in the source why not remove it from the source using ME or QC or even a new bespoke Kobo-centric epub modifying plugin?

For the "calibre cruft" (very subjective), won't the calibre send-to-device process put it all back in again in the on-device kepub? If you're talking about removing it only from the calibre library kepub then that might be useful if you wanted to manually edit the kepub because the OPF file is very unwieldy with all the custom metadata included in there. OTOH only a masochist would want to manually edit a kepub because of all those KoboSpans - even if you could open a .kepub file extension in the calibre Editor - which currently you can't via the GUI.

[geek1011]

Originally Posted by jackie_w

This is where my 'can of worms' sensor went off...

... seems to allow for auto-incorporating what would normally be applied via a patch.

Yes, for the full screen patch, but many people were irrationally afraid of patching. You should see some email discussions I had... As for the CSS inlining, this cannot be fixed in a patch.

... seems to be a prime candidate for conflicting with the 'Modify CSS' option in the KT/KTE drivers and the Hyphenation option in KTE.

You do realize kepubify is originally made to be used standalone without Calibre. Just because kepubify has a feature doesn't mean I'll expose it to the Calibre plugin.

If there are a well-defined set of problems with a source book why fix the kepub? Why not fix the source instead? That's what the Modify Epub (ME) and Quality Check (QC) plugins were supposed to be for.

A few reasons:
1. Some only affect the Kepub renderer
2. Too many people complained (I'm looking at you, self-closing title/script tags...)
3. Clearing up those issues during conversion helps eliminate them as sources of issues
4. Many people are lazy/don't know how/can't fix or recognize these issues

... again, if there is MS/ADE/iBooks cruft in the source why not remove it from the source using ME or QC or even a new bespoke Kobo-centric epub modifying plugin?

Again, those don't need to be included as an option in the plugin, and kepubify is meant to be used standalone.

For the "calibre cruft" (very subjective), won't the calibre send-to-device process put it all back in again in the on-device kepub? If you're talking about removing it only from the calibre library kepub then that might be useful if you wanted to manually edit the kepub because the OPF file is very unwieldy with all the custom metadata included in there. OTOH only a masochist would want to manually edit a kepub because of all those KoboSpans - even if you could open a .kepub file extension in the calibre Editor - which currently you can't via the GUI.

I'm crazily nitpicky about extra metadata/CSS classes/elements/nonsense in my ebooks, but that's just me.

[jackie_w]

Originally Posted by geek1011

You do realize kepubify is originally made to be used standalone without Calibre.

Of course I realise that! I'm not completely clueless I don't believe I've ever expressed any concerns whatsoever about anything included in standalone kepubify. Standalone is standalone ...

Originally Posted by geek1011

Just because kepubify has a feature doesn't mean I'll expose it to the Calibre plugin.

... OTOH your earlier post was not at all clear (at least to me) about which current features you planned to include in a kepubify calibre plugin. Why would I not think you may be planning to include everything unless specifically excluded? Such as when you said:

The metadata handling all will be in seriesmeta (an add-on tool for kepubify), which is a standalone tool which will not be included in the Calibre plugin.

Anyway, I've said enough for now. It was not my intention to pick a fight.

[geek1011]

Originally Posted by jackie_w

Of course I realise that! I'm not completely clueless I don't believe I've ever expressed any concerns whatsoever about anything included in standalone kepubify. Standalone is standalone ...

Of course I should have realized that . I was making sure we were on the same page.

... OTOH your earlier post was not at all clear (at least to me) about which current features you planned to include in a kepubify calibre plugin. Why would I not think you may be planning to include everything unless specifically excluded? Such as when you said:

Yes, I didn't clarify that well enough. I only intend to include features which will not conflict with features the driver has which are applied when sending an existing kepub to the device. I'll need to look into this as I implement it.

Anyway, I've said enough for now. It was not my intention to pick a fight.

Lol, your opinion's always welcome.

This thread has really degenerated from a discussion about kepubs to kepub conversion tools to the upcoming kepubify Calibre plugin though...

[davidfor]

Originally Posted by geek1011

Mainly speed, stability for large books, and some of the built-in CSS mods (such as the full screen fixes until recently, the CSS inliner to deal with the buggy FW and some buggy books, etc).

I have a well-defined scope for kepubify. Kepubify (the converter part only) will only:
- Convert epubs to kepubs
- Apply fixes to book-breaking (i.e. book is not readable) CSS and HTML layout which are in multiple books by a publisher
- Remove extra MS/ADE/iBooks/Calibre cruft
- Apply CSS modifications to fix or tweak bugs (see above) or Kepub features (like hyphenation)

.

I don't think I have seen any mention of most of that before. And I don't know what firmware bugs you are fixing. For example, I cannot remember mention of a bug in kepubs that needed the CSS to be inlined to fixed. And what are the other changes? You don't mention anything about them on the kepubify page. Is there a list somewhere of what is done? And how is the custom CSS handled? Added to each stylesheet, as a separate stylesheet or internal to the page? Does the stylesheet inlining do the latter or add stylesheets to each tag?

Some of this I could see by running kepubify with the different options. But, others would need books with those problems to see. But, as the kepubify page doesn't mention some of this, I wouldn't even be looking for it.

[geek1011]

Originally Posted by davidfor

I don't think I have seen any mention of most of that before.

I've mentioned it in bits and pieces scattered around many places. I'll be consolidating it for kepubify v3.

And I don't know what firmware bugs you are fixing. For example, I cannot remember mention of a bug in kepubs that needed the CSS to be inlined to fixed.

https://www.mobileread.com/forums/sh....php?p=3769235

And what are the other changes? You don't mention anything about them on the kepubify page. Is there a list somewhere of what is done?

Most are in the command documentation, some I've removed after the need was gone. You can find the most up-to-date information in: https://github.com/geek1011/kepubify...kepub/kepub.go

And how is the custom CSS handled? Added to each stylesheet, as a separate stylesheet or internal to the page? Does the stylesheet inlining do the latter or add stylesheets to each tag?

Custom CSS is currently added before the body tag end after all other CSS tweaks. Inlining adds the stylesheet as a style tag in the same place the link tag was (to preserve precedence).

Some of this I could see by running kepubify with the different options. But, others would need books with those problems to see. But, as the kepubify page doesn't mention some of this, I wouldn't even be looking for it.

That's partly the intention. You aren't supposed to notice the fixes unless they are necessary. But, I agree they need to be documented properly. That will be resolved in kepubify v3.

[davidfor]

Originally Posted by geek1011

I've mentioned it in bits and pieces scattered around many places. I'll be consolidating it for kepubify v3.


https://www.mobileread.com/forums/sh....php?p=3769235

Now I remember. But, that was a problem that existed in one firmware version that was replaced after three days. If it doesn't fix anything else, it isn't a particularly useful option.

Most are in the command documentation,

I have to disagree with that. I was looking at your page for kepubify when I wrote my post. The lack is what prompted my comments.

some I've removed after the need was gone. You can find the most up-to-date information in: https://github.com/geek1011/kepubify...kepub/kepub.go

Yes, I can read code, but most can't or won't. And nothing in the code tells me why the changes exist. Is it fixing a firmware bug, a book bug, or because that's the way you think it should be.

That's partly the intention. You aren't supposed to notice the fixes unless they are necessary. But, I agree they need to be documented properly. That will be resolved in kepubify v3.

I think documenting the bug workarounds are almost more important than the other things. If we don't know what is being fixed, how do we know what to look for in problems. And this is full of, to me, unexpected side-effects. Nothing hints that you are doing the smarten punctuation. Or the removal of the "Replacement character". Or replacing the unicode non-breaking space with an HTML entity.

Improving the documentation in v3 is a good idea.

And for the record, I'm terrible at documentation and know I write it from the point of view of the developer. I keep hoping someone will point out the problems (missing, unclear, errors or anything) but it rarely happens

[forster01]

i have a epub, with two embedded .ttf fonts, one is used only for chapters, the other one is used for a first few words in the beginning of the each chapter.

in the epub, everything is displayed fine. In the kepub chapter font is not visible, but the other one is visible... i think it's a bug in new 4.12 firmware

[jackie_w]

Originally Posted by forster01

i think it's a bug in new 4.12 firmware

Probably not. It's more likely that it's a combination of the way that particular book has coded its HTML/CSS and the aggressive overrides the kepub renderer uses in order to try and enforce your chosen settings in your device's font menu. The epub renderer uses less aggressive override rules.

It's almost certainly fixable but it depends whether you want to do the work.

Solving CSS overriding problems like this (font-family, line spacing, text alignment) usually involve a combination of manually editing the HTML/CSS for that particular book and/or making the kepub override rules less aggressive on your Kobo. The latter can be done via various Kobo patches but as this will affect all your books afterwards it may have unwelcome consequences for some books if/when your font menu choices are not enforced when you want them to be.

[geek1011]

Originally Posted by davidfor

Now I remember. But, that was a problem that existed in one firmware version that was replaced after three days. If it doesn't fix anything else, it isn't a particularly useful option.

According to some emails I have recieved (I haven't checked this myself), it has helped with certain ebooks which don't show their styles otherwise. As for me, I find it a useful debugging option in certain cases.

Yes, I can read code, but most can't or won't. And nothing in the code tells me why the changes exist. Is it fixing a firmware bug, a book bug, or because that's the way you think it should be.

Most optional changes are for firmware bugs or customization, most unusual mandatory changes are to fix issues reported to me about differences in the kepub and epub renderer. The nbsp replacement is one of these (they don't show otherwise). The smart punctuation is my opinion (it isn't as heavy handed as others though), and the others are just code cleanups which make no difference otherwise (like the ms tags and the indentation of the opf). I'll work on documenting them all as I work on v3.

I beg to differ on that (but I give you the smart punctuation one). The replacement character is to fix char encoding bugs, and makes no difference otherwise. Not doing the nbsp replacement causes side effects in itself.

And for the record, I'm terrible at documentation and know I write it from the point of view of the developer. I keep hoping someone will point out the problems (missing, unclear, errors or anything) but it rarely happens

Reminds me of myself . I have to work hard not to write like that, but I find it hard as I have to consciously hide certain details which would be relevant to developers. That's why I kept giving up on writing kobopatch documentation (but I'll get around to it sometime...).

Interesting discussions to me at least, but yes it has meandered a bit.

Bringing it back to original subject, can we agree that here are the main takeaways for ordinary Kobo/Calibre users (folks who are not devs) trying to get the most out of their Kobo device?

1. To get full functionality out of your Kobo, it's helpful to convert the books on it to the .kepub format. Kepub is just a slightly optimized form of .epub.
2. Converting to .kepub gives you at least these features:
- Faster, more responsive page turns.
- Better image handling, including display, zooming, and panning.
- Footnote preview (footnote link pops up in the page you're on, rather than going to another page)
- Better epub 3 feature support
- Better, more responsive text selection for copying, bookmarking, and annotations.
- Fast page flip (press lower corner, pages flip fast till you stop)
- Reading statistics
3. To convert in Calibre, just install the KoboTouchExtended plugin (developed by @jgoguen), which is not installed by default. That's it, no need to do anything else. Sync your non-DRM'd books from Calibre to your device, and Calibre does the work.
4. If you want to convert your books to .kepub without using Calibre for some reason, there's a command line utility called kepubify (developed by @geek1011).

Thanks all.

[forster01]

Originally Posted by jackie_w

Probably not. It's more likely that it's a combination of the way that particular book has coded its HTML/CSS and the aggressive overrides the kepub renderer uses in order to try and enforce your chosen settings in your device's font menu. The epub renderer uses less aggressive override rules.

It's almost certainly fixable but it depends whether you want to do the work.

Solving CSS overriding problems like this (font-family, line spacing, text alignment) usually involve a combination of manually editing the HTML/CSS for that particular book and/or making the kepub override rules less aggressive on your Kobo. The latter can be done via various Kobo patches but as this will affect all your books afterwards it may have unwelcome consequences for some books if/when your font menu choices are not enforced when you want them to be.

Apparently it is a bug. When a font is defined in for example h1 class it won't be shown. It will be visible only as span class font-family. In kepub that is.

[forster01]

Here is what i meant: epub/kepub embed test
Font used:"Top Secret"
Font source: Font Squirrel
Kepub by kepubify

[jackie_w]

Originally Posted by forster01

Apparently it is a bug. When a font is defined in for example h1 class it won't be shown. It will be visible only as span class font-family. In kepub that is.

I still disagree. It's a Kobo design decision for font overriding that you don't like. As it happens I don't like it either, which is why I use the patches to avoid this kind of problem.

FYI the font-family override rule for kepubs is

* { font-family: %1 !important; }

which is just about as aggressive as it gets. The Adobe renderer equivalent for epubs is

@font-face { font-family: -ua-default; font-style: normal; font-weight: normal; src: url('res:///fonts/normal/%1') }
@font-face { font-family: -ua-default; font-style: italic; font-weight: normal; src: url('res:///fonts/italic/%1') }
@font-face { font-family: -ua-default; font-style: italic; font-weight: bold; src: url('res:///fonts/bolditalic/%1') }
@font-face { font-family: -ua-default; font-style: normal; font-weight: bold; src: url('res:///fonts/bold/%1') }
body, p { font-family: -ua-default !important; }

If you don't patch, to see embedded fonts you're supposed to always select 'Publisher Default' from the font menu? What happened when you did that? Your kepub example works on my Forma with Publisher Default but as mine is patched I can't draw any conclusions.

[forster01]

as i said i don't have font on h1

i'm struggling to make it work all day