EPUB, KEBUP, ShmEPUB... confused. What's on my reader now?

[Katsunami]

My base format in my library is EPUB, and have been converting (without any problems) to MOBI and later AZW3 in the past, as I've owned two Kindles.

I've got a KA1 since yesterday, and have been trying various things to get as much usable screen space as possible, without losing functionality. It looked like I would be using KEPUB.

1. I have added the full screen reading option:

[FeatureSettings]
FullScreenReading=true

2. I installed the Kobo Touch Extended driver plugin, the KEPUB metadata plugins, and the KEPUB output plugin in Calibre.

3. I've converted some books to KEPUB, and uploaded them to the reader.

Now I've found out that the old letter cut-off bug still exists in the full screen mode, and no amount of tinkering with CSS can fix it (and I've tried a lot of stuff I found around the internet.)

In the end, I removed the KEPUB conversions and decided to see what EPUB would bring. In the Kobo Touch Extended driver, I unticked every format, except EPUB. In the "Extended" tab, the only option that is ticked is Enable Extended Kobo Features.

When I now upload a book to the KA1 (without first converting it to KEPUB with the output plugin), it displays perfectly in full screen mode; no letters are cut off. However, when I display the header and footer, I see the ones as displayed by the KEPUB reader, not the EPUB one which only has a footer, AFAIK. Also, the reading progression functions such as Time To Read seem to be working, which I was led to believe to only work with KEPUB.

The files on the reader are "filename.kepub.epub".

So, what is on my reader now?

Is it an EPUB, displayed by the KEPUB reader, because of the ".kepub.epub" extension?

Is it a KEPUB, created by the Kobo Touch Extended driver? I've read the driver can convert files on the fly before uploading them. If it does, it must be doing something different to the KEPUB output plugin, because the file as uploaded by the driver displays fine, while the file created by the KEPUB output plugin has the last letter on each line cut off.

Is it a ShmEPUB... an EPUB or KEPUB that is fixed or has other non-standard things done to it by the driver such as the injection of some CSS, to display correctly in full screen mode?

Thanks in advance

[jackie_w]

@Katsunami,

If you are using KoboTouchExtended plugin and you have ticked Enable Extended Kobo Features then your calibre standard epubs have been converted to kepub during send-to-device.

If you want standard epub you can either switch back to the default KoboTouch driver or keep the KoboTouchExtended plugin driver and untick Enable Extended Kobo Features.

I was not aware that that either of the drivers fixed the kepub letter cut-off bug (this problem doesn't exist for standard epubs). It can be done via one of the Kobo Patcher patches but I believe you've previously expressed reluctance to try patching.

[GeoffR]

If the book shows "KOBO EPUB" in the device's library listing it is a KePub. If it just shows "EPUB" then it is an ePub.

For sideloaded KePubs the KePub full-screen text cut-off problems can be fixed without a patch if you set up the KoboTouchExtended driver correctly, but I don't know about the KePub Output plugin.

There are two things that need to be done to the KePub to fix the problems:

1. A suitable wrapper div needs to be added to each html file. This is done automatically by the KoboTouchExtended driver, but I don't know if the KePub Output plugin does this.

2. The KePub stylesheet needs to be modified to add:

body { margin: 0 !important; padding: 0 !important; }
div#book-columns { padding: 0 0.2em !important; }

The KoboTouchExtended driver can be set up to add those to KePub stylesheets automatically, but it doesn't add them by default. I don't know if the KePub Output plugin can do this.

(The above applies to horizontal languages, use div#book-columns { padding 0.2em 0 !important; } for vertical languages such as Japanese.)

Edit: For details of how to set up the KoboTouchExtended driver to add the above CSS to KePubs automatically, see https://github.com/jgoguen/calibre-k...-arbitrary-css

[Katsunami]

Thanks for the answers

Originally Posted by jackie_w

I was not aware that that either of the drivers fixed the kepub letter cut-off bug.

It doesn't. I've been testing with several books, and sometimes, it happens that a book is displayed correctly when first opened and then later reverts to show this bug. Most books show this bug immediately when opened. Some books don't seem to be affected by it at all.

It can be done via one of the Kobo Patcher patches but I believe you've previously expressed reluctance to try patching.

Yes; normally I don't really mind patching or updating software, but that is only for devices I can have full control over. I don't know enough about e-readers and their operating systems to confidently patch the firmware *and* be sure that updates/upgrades keep working as they should.

Originally Posted by GeoffR

1. A suitable wrapper div needs to be added to each html file. This is done automatically by the KoboTouchExtended driver, but I don't know if the KePub Output plugin does this.

IS the "book-columns" the intended wrapper? If so, it can probably be checked with the editor or in the KEPUB plugin code to see if it added.

2. The KePub stylesheet needs to be modified to add:

body { margin: 0 !important; padding: 0 !important; }
div#book-columns { padding: 0 0.2em !important; }

The KoboTouchExtended driver can be set up to add those to KePub stylesheets automatically, but it doesn't add them by default. I don't know if the KePub Output plugin can do this.

Thanks. I've tried the "margin" line, but haven't encountered or missed the red book-columns line. I know how to set up the Kobo Touch Extended Plugin to add stylesheets; I'll redo it, and see if this fixes the problem.

Thanks for taking the time to answer

[GeoffR]

Originally Posted by Katsunami

IS the "book-columns" the intended wrapper? If so, it can probably be checked with the editor or in the KEPUB plugin code to see if it added.

Yes,if the wrapperdivs are being added the body of the html will look something like:

<body>
  <div id="book-columns">
  ...
  </div>
</body>

Thanks. I've tried the "margin" line, but haven't encountered or missed the red book-columns line. I know how to set up the Kobo Touch Extended Plugin to add stylesheets; I'll redo it, and see if this fixes the problem.

There are really two seperate problems: The first is where the whole page is shifted to the right and all text is cut off at the right margin; this one is solved by setting the body margin/padding to zero. The second is often not very obvious, it really depends on which font you use, it is where certain letters with large side bearing values are cut off when they appear at the beginning or end of a line, such as J at the beginning or f at the end; that one is fixed by adding the padding to the wrapper div.

[Katsunami]

Thanks for the information. Tonight I'll set up the Kobo Touch Extended Driver to take this into account.

I don't really mind what format the reader is using. My base format is EPUB, and I meticulously edit/fix every book I add to the library. (Except, apparently, setting body margins and paddings to 0.) On converting, I apply modifications needed for the particular reader, which for the KPW1, are none.

[Katsunami]

Oh, by the way... It just dawned on me that the kobo_extra.css file for the Kobo Touch Extended driver probably has to have Unix/Linux line-endings? I'm quite sure I didn't pay attention to that when I first made that file to try and fix this font cut-off issue.

The instructions on Github are different than the ones in the driver. They say a kobo_extra_<device>.css needs to be created in the plugin folder (the KA1 is not yet listed), while the driver itself says that "kobo_extra.css" needs to be put into the root of the device.

If I'm inferring this correctly, the kobo_extra_<device>.css files can be made to make the driver support different devices at the same time, each with their own CSS, and it'll put them onto the Kobo device as "kobo_extra.css".

Edit: In the end, it doesn't work.

I've added the two CSS lines to "kobo_extra.css" in the root of the reader's main memory, and set line endings to Unix/Linux. I'm sending the files using the Kobo Touch Extended driver with the "modify css" option selected. I've downloaded one of the books from the device and checked in the Calibre Editor: the file "stylehacks.css" has been added and linked in the html files, and contains the lines above, so everything is working as it should; this hack doesn't solve the letter cut-off problem on the Aura ONE.

[PeterT]

I don't think line endings make any difference here at all.

[Katsunami]

It might be fixed. I'm now testing. I've used information from this thread, and changed this line:

div#book-columns { padding: 0 0.2em !important; }

to:

div#book-columns div { padding-left: 2% !important; padding-right: 2% !important; }

Note that there is an extra div (in red), and that I'm now using full css properties instead of shorthand.

This *seems* to have fixed the problem with at least two books, but the margins are quite big, even with the setting at the very lowest. I'm now testing with things such as 1em and lower... to see how low we can go... limbo-rocking css....

edit: I'm now down to a padding of 0.1em, and nothing seems to have changed from 0.25em. I'll test with a padding of 0, and see what happens; maybe explicitly setting SOME padding is already good enough, so the default isn't being used.

[John F]

Katsunami, any chance you could attach an epub (not a kepub) you are having problems with? I would like to convert it and see if I have the same problem.

[Katsunami]

Sorry, I can't. All of my books are copyrighted. If I attach one of them here, I'll probably get banned for uploading copyrighted material. The few that are not copyrighted don't have any problems, because I created them myself from Gutenberg texts, and the code is thus hand-made and (almost... ) perfect.

edit: It looks like the problem is fixed. It also seems that, if the padding gets below a certain value, nothing changes. I have created a file "kobo_extra.css" in the root of the KA1, with Unix/Linux endings, and added the following:

body { margin: 0 !important; padding: 0 !important; }
div#book-columns div { padding-left: 0 !important; padding-right: 0 !important; }

The margins around the text, when set at the lowest settings (with the books I've tested up until now) are comparable to the ones on my Kindle Paperwhite. The margins are maybe one character at each side or so. With these settings, and at least these books, it's not possible to extend the text all the way to the edges.

I don't mind about that, because I don't want text right up to the edge, so the margins are fine with me. Even if you lose a character left and right, and about half a line on the top and bottom, the loss is MUCH less than with the standard view that displays headers and footers.

If this fixes the full screen view for all of my books like this, I'm very happy, and the KA1 will be a *HUGE* upgrade to my KPW1. (With the default view, it would have been a large upgrade only.... )

To make sure, I'll test without kobo_extra.css, and also test what will happen if I first convert using the output plugin and then send the books to the device. One moment...

edit: weird. The KA1 seems to be caching some things. At first, when I was lowering the padding and removing/re-uploading the books, nothing seemed to change. Even at 0, nothing changed; the problem stayed fixed, with the same margin widths. Then I started opening/switching books, and suddenly the problem rose again.

edit 2: nothing works anymore. 1em, 0, 0.2em... nothing makes a difference. On some books, the margin slider actually doesn't work anymore. I'm going to try with 2% again, and see if that helps.

[Katsunami]

It's no use trying to fix this using CSS. It doesn't work on the Aura ONE.

When in a chapter of the book, the problem can be fixed by disabling the full screen mode, and then sliding the margin slider to a different location, and back again. This will fix the screen *regardless* of what is in the kobo_extra.css file. If you then switch back to the full screen mode, the display stays fixed, until you move to something that is rendered differently than a chapter, such as the inline table of contents. If you then move back to a chapter again, the problem returns.

The reason why I thought it was fixed was because I was jumping straight into chapters from the reader's table of contents after accidentally 'fixing' the layout first. I can now reliably make the problem reappear and disappear, regardless of what is in the CSS-file, at least with the books I've been testing.

[GeoffR]

The line endings in the CSS file make no difference, it is Calibre that reads the CSS file not the Kobo device.

Perhaps you could find a small public domain book that has the problem, and post a copy of the KePub that Calibre has sent to the device, i.e. use calibre to send the book to your device and then copy it off the device and post it here.

Also you might post a screen shot that shows the problem, it could be that we are talking about different things (e.g. there is another problem that affects KePubs where the text is cut off at the bottom of the screen, that problem cannot be fixed by CSS.)

In the end there is always the option of sending the book as an ePub. ePubs don't suffer the cut-off problems that afflict KePubs in full-screen mode.

Edit: Of course it could be a new bug in the new firmware too, I am not running the new firmware, but if you post a book I could test it on the old firmware to see if the problem is the book or the firmware.

Edit2: Also to note: there have sometimes been problems that occur if you switch between normal reading mode and full-screen mode from within a book, it is best to exit the book, switch reading modes, then open the book again. There is also a KePub bug that started in the 3.19 firmware where the margin slider stops working, for which the work-around it to exit the book and open it again. None of these problems seem to affect ePubs though.

[Katsunami]

Originally Posted by GeoffR

Edit2: Also to note: there have sometimes been problems that occur if you switch between normal reading mode and full-screen mode from within a book, it is best to exit the book, switch reading modes, then open the book again.

I'll try that. I've noticed that most of the time, the problem reasserts itself as soon as the view switches from something that is not a chapter (such as an inline table of contents, or a title page), and a normal chapter with block text.

There is also a KePub bug that started in the 3.19 firmware where the margin slider stops working, for which the work-around it to exit the book and open it again. None of these problems seem to affect ePubs though.

I've noticed this as well. At first I thought the problem was caused by the addition to the CSS, just like on some mobile sites, the CSS-code prevents the site from being resized.

EPUB has it's own problems though. First, you can't use all the new features such as time to read, and chapter preview (I wonder why Kobo had to go and create their own bastard EPUB version; tablet programs can do that sort of thing with regular EPUB's). Second, there seems to still be a bug that leaves a part of the screen blank; sometimes even half the screen. I tried to solve it with a orphans/widows-setting of 0, but that basically destroys the entire EPUB-rendering engine on the KA1. The books end up as if they have no markup at all, at least in the few books I've tested.

I'll first test a rendering mode switch and then opening a book to see if that fixes the problems; if I can, I'll try and see if I can find a public domain book that has this problem.

[GeoffR]

Originally Posted by Katsunami

Second, there seems to still be a bug that leaves a part of the screen blank; sometimes even half the screen. I tried to solve it with a orphans/widows-setting of 0, but that basically destroys the entire EPUB-rendering engine on the KA1. The books end up as if they have no markup at all, at least in the few books I've tested.

I set widows:1;orphans:1; in all my books (orphans:0;widow:0 works just as well) and have never had any problem like that.

Usually if a book appears as if it has no stylesheet at all, it means that there was a fatal error in the stylesheet. The ePub reader is very sensitive to errors and will just discard the whole stylesheet if it is not perfect, so it can be a good idea to run epubcheck after making any manual changes.