Using calibre.ebooks.oeb.polish.container in a driver

[davidfor]

One of the suggestions for the Kobo driver has been a way to tweak the stylesheet during the send to device. There are a few things that Kobo does that would be nice to change. And while they could be edited into the ePubs, that means they are there for other devices which might not be desirable. The intention is to have a stylesheet in the root directory of the device and append that to any stylesheets in the ePub.

In fact, someone has written the code as a subclass of the KoboTouch driver and given me a copy. It works well and I am merging it into the KoboTouch driver. One thing it uses calibre.ebooks.oeb.polish.container. It has all the function necessary to find and modify the stylesheets.

But, I am getting an error when I run calibe from source. If I import the container where I use it, everything is OK. But, that means it will get imported each time a book is sent, which I assume will have a performance hit. If I put it at the top with the other imports, I get an error during startup when running from source. The statement added to the Kobo driver code is:

from calibre.ebooks.oeb.polish.container import get_container

The error is:

Python function terminated unexpectedly
  cannot import name plugin_for_input_format (Error Code: 1)
Traceback (most recent call last):
  File "site.py", line 132, in main
  File "site.py", line 109, in run_entry_point
  File "D:\Development\GitHub\calibre\src\calibre\debug.py", line 213, in main
    from calibre.gui2.main import main
  File "D:\Development\GitHub\calibre\src\calibre\gui2\main.py", line 18, in <module>
    from calibre.library.database2 import LibraryDatabase2
  File "D:\Development\GitHub\calibre\src\calibre\library\database2.py", line 30, in <module>
    from calibre.customize.ui import (run_plugins_on_import,
  File "D:\Development\GitHub\calibre\src\calibre\customize\ui.py", line 15, in <module>
    from calibre.customize.builtins import plugins as builtin_plugins
  File "D:\Development\GitHub\calibre\src\calibre\customize\builtins.py", line 669, in <module>
    from calibre.devices.kobo.driver import KOBO, KOBOTOUCH
  File "D:\Development\GitHub\calibre\src\calibre\devices\kobo\driver.py", line 30, in <module>
    from calibre.ebooks.oeb.polish.container import get_container
  File "D:\Development\GitHub\calibre\src\calibre\ebooks\oeb\polish\container.py", line 19, in <module>
    from calibre.customize.ui import (plugin_for_input_format,
ImportError: cannot import name plugin_for_input_format

I am assuming I have missed something, but can't see what it is. Can anyone tell me?

Of course, if there is another way of doing what I want that I have missed, that would be even better. I had a look, but couldn't find anything.

[kovidgoyal]

That's likely a circular import, dont put imports at the top level, they slow down calibre startup. Generally re-importing an already imported module is fast, since it basically does a dict lookup and re-uses the imported copy. Certainly that time will be negligible compared to the time required to edit the epub.

[davidfor]

A circular import was what my google searches suggested as well. The bit that had me was I did the same thing in a user plugin and saw the error when running from source but didn't get an error when running from the install. I'll put it down to a difference in the debug vs production environment.

Using the user plugin version, I have done a performance test. For 933 epubs, it took an extra 7 seconds when the import was just before use. As far as I'm concerned, that's negligible.

[kovidgoyal]

You can also do the import in the open method and assign the open object as an attribute of the driver which can then be used later.

[davidfor]

Thanks to all this, I nearly have it all working. But, I discovered today that the CSS parsing runs CSSPreProcessor which strips the @page rules. I can see why that was done, but it's a nuisance for me. For the moment, I'm setting the preprocessor to a dummy one that simply returns the CSS without changes.

[kovidgoyal]

Polish books does not modify existing css (only adds new css if needed), therefore the css parsing is intended to be used read-only. The existing css preprocessor is designed for the conversion pipeline, not polishing. You should be fine with using a dummy version.

[kovidgoyal]

I have added some documentation to the parse_css method warning that it should be overridden if you intend to write back the css.

[davidfor]

Originally Posted by kovidgoyal

Polish books does not modify existing css (only adds new css if needed), therefore the css parsing is intended to be used read-only. The existing css preprocessor is designed for the conversion pipeline, not polishing. You should be fine with using a dummy version.

In that case there is a problem. I couldn't think of why Polish books would modify the CSS, so I had a look. Subsetting embedded fonts was the first spot that I found. When I used that, it removed the @page rules from my stylesheet. I also checked by doing a conversion and adding margins to the page. Again, Polish books removed the @page rules when I subsetted the fonts.

I got caught when doing the above tests. I was surprised that Polish books processed the original_epub. I understand why the conversion does this, but I was expecting Polish books to polish the final product.

[kovidgoyal]

Ah yes, there is one case where css can be modified, if subsetting fonts removes a font because it is unused, it's corresponding font face rules are removed. I will fix that for the next release.

[kovidgoyal]

https://github.com/kovidgoyal/calibr...231f23914aab5b

[davidfor]

I can work with that. I will still override the preprocessor but I'm tempted to leave it and kill the bad MS stuff if it is there.