Converting to PDF with Prince

[Jellby]

I am considering porting my Prince conversion utility to a calibre plugin. My problem is I'm very much a python (and calibre) newbie, so this will not be a trivial work for me.

I've had a look at the plugin documentation, but there are some details about what I would need that are not available. For example, how can I obtain a list of the filenames in a book and in the spine? And is there some precooked solution to have the files in a named temporary location so they can be fed to Prince?

My original code is actually very simple: just uncompress an epub, add some custom CSS files and issue a command for Prince containing all filenames from the spine.

Another question is whether this should be a "general" plugin or an "output" plugin, given that it is using an external non-free tool and it would replace the already existing PDF conversion.

[kovidgoyal]

Is there something that prince does that calibre's own conversion to pdf does not do?

[Jellby]

Originally Posted by kovidgoyal

Is there something that prince does that calibre's own conversion to pdf does not do?

I haven't tested calibre's conversion much, but Prince is supposed to have better typesetting than normal HTML renderers (in particular better paragraph-breaking algorithm). It also supports a number of CSS3 and custom CSS properties allowing some interesting things like real floating images (no white space when an image is moved to the next page), real footnotes (although most of the styling is missing), special page styles (headers/footers) for the first page of a chapter, etc.

But probably the main reason for my tool is that it allows to use specific CSS stylesheets only for the PDF conversion. One additional stylesheet may contain user preferences, like page size, fonts (already available in the calibre configuration), but also margins, justification, hyphenation limits, etc. (that's a suggestion for calibre). Another stylesheet may be included in the book itself (in a <meta> tag), and it's intended to have some some styles to work specifically with Prince, using the above features for example.

I've just compared both PDF conversions, and it looks like the kerning from calibre is not that good either. I've attached the results, you can compare yourself. There are also differences in the title page picture (a problem of default resolution probably), in the TOC (Prince added the page numbers to the HTML TOC, I guess calibre can only do that if it generates the TOC itself), in the PDF bookmarks (calibre has only one level), in the <hr> (compare page 41)... The comparison is not 100% fair, because this book is already tweaked for Prince (with the special style I mentioned above).

[kovidgoyal]

The kerning is indeed off in the calibre pdf, I suspect that has to do with the font used, Qt is rather finicky about what fonts it renders well. calibre is perfectly capable of generating multi-level tocs, you just need to set the appropriate conversion options. You can also control margins, justification and set a special stylesheet. And calibre uses the Qt WebKit rendering engine, which means it's support for CSS is likely to be pretty good

Anyway, I was just curious. Regarding your original questions, if you make it a standalone plugin you will need to do more work --- get the file from the database, explode the epub yourself and so on. SO make it a conversionplugin.

The conversion plugin is called witht eh book already exploded into its individual files, in an "oeb" object. Just have a look at one of the simpler conversion output plugins in calibre to see how to use the oeb object. For example, the rtf or txt output plugins. Or the htmlz output plugin. And feel free to ask if you need clarification.

[Jellby]

Originally Posted by kovidgoyal

Just have a look at one of the simpler conversion output plugins in calibre to see how to use the oeb object. For example, the rtf or txt output plugins. Or the htmlz output plugin.

Where can I find the source for those plugins?

By running a dummy plugin with debugging, I see this in the terminal:

Trimming unused files from manifest...

Is it possible to avoid that? I'd have to use one of these files for the conversion...

[kovidgoyal]

Looking in ebooks/conversion/plugins. I dont see how you could avoid the trimming of unused files. If you want to pass in some special css to prince, you can make an option in the output plugin and pass in the css that way, in any case the conversion pipeline will remove any css that is not used by some html.

[Jellby]

Originally Posted by kovidgoyal

Looking in ebooks/conversion/plugins.

I was just going to write that I found them

I dont see how you could avoid the trimming of unused files. If you want to pass in some special css to prince, you can make an option in the output plugin and pass in the css that way, in any case the conversion pipeline will remove any css that is not used by some html.

The problem here is that this would be a different (optional) CSS file per book. I include such a file with my ePubs. I could maybe use @media rules, but still, this is a stylesheet that's only intended to be used with Prince...

I guess I'll have to go the long way, explode the file and parse the opf.

[Jellby]

I tried this:

tdir = PersistentTemporaryDirectory('_PrincePDF')
exploder = get_tools(format)[0]
opf = exploder(book, tdir)
#os.chdir(os.path.dirname(opf))
oeb = OEBBook(default_log, HTMLPreProcessor, CSSPreProcessor)
OEBReader()(oeb, opf)

(where "format" and "book" are arguments passed to the function".) The exploding seems to go fine, if I print the "tdir" value I can see that the directory is created and all the ePub files are there. I have tried with and without the chdir, but I always get:

Failed to parse content in Chapitre-36.xhtml
Failed to parse content in Chapitre-14.xhtml
Failed to parse content in Chapitre-21.xhtml
[...]
Referenced file u'Chapitre-24.xhtml' not in manifest
Referenced file u'Chapitre-12.xhtml' not in manifest
Referenced file u'Chapitre-10.xhtml' not in manifest
[...]
OEBError: Spine is empty

What am I doing wrong so far?

EDIT: Solved, I had to use preprocessor instances:

html_preprocessor = HTMLPreProcessor()
css_preprocessor = CSSPreProcessor()
oeb = OEBBook(default_log, html_preprocessor, css_preprocessor)

Now the unused files are not removed from the manifest, but the metadata does not correspond to the OPF (illustrator or translator do not appear, neither do <meta> tags). In my tool I was using "import xml.dom.minidom", is that safe to use within a calibre plugin? Is there any alternative?

[kovidgoyal]

If you wan to parse the OPF use lxml. You can use xml.dom.minidom, but lxml is better. calibre will only read metadata it supports itself when parsing the opf.

[Jellby]

Thank you. First version is working, now I have to add all the options, config and GUI.

[Jellby]

OK, here is a more or less working version (no help message yet, and default demo icon).

Some issues I have:

- The "View log" does not look disabled, although I use setEnabled(False). It's not clickable by default, but at least in my computer it's not greyed-out or anything.

- I don't know what to do with the print statements. I'd like them to be off by default, and only appear with some minimal debug level. Better yet, I'd like to have some status indication... can I use some calibre function for that or should I have it inside the plugin?

- In the configuration dialog, is there any way to have the "Restore defaults" button together with OK and Cancel?

- The output PDF is still created in a temp dir, I'd like to add an option to choose whether it should be added to the library or saved to disk elsewhere. Can I use any calibre functions for that? In one case, I'd add a file to the existing book record, and I guess calibre would take care of everything. In the other case I'd probably like to use the "save to disk" template at least.

[kovidgoyal]

from calibre.constants import DEBUG
if DEBUG:
print (whatever)

then prints will only happen when running calibre in debug mode.

See the documentation for QDialogButtonBox, you can add as many types of buttons as you like.


db = self.gui.current_db
db.new_api.add_format(book_id, 'pdf', path)

If you want to use save to disk templates, use the get_components() function from library/save_to_disk.py

[Jellby]

Originally Posted by kovidgoyal

from calibre.constants import DEBUG
if DEBUG:
print (whatever)

Great! Is there something that will work with 'calibre -v' too? What about some GUI feedback?

See the documentation for QDialogButtonBox, you can add as many types of buttons as you like.

But I didn't define the QDialogButtonBox for the config dialog. As far as I can see that is defined in calibre/customize/__init__.py:

button_box = QDialogButtonBox(QDialogButtonBox.Ok | QDialogButtonBox.Cancel)

Can I access from my plugin the QDialogButtonBox that is defined in the standard do_user_config procedure?

[kovidgoyal]

There is no calibre -v, it's calibre-debug -g

If you want to implement anything beyond the most basic customization you have to implement your own config widget, http://manual.calibre-ebook.com/crea...of-your-plugin

[Jellby]

Originally Posted by kovidgoyal

There is no calibre -v, it's calibre-debug -g[/url]

I mean 'calibre --verbose', as explained in http://manual.calibre-ebook.com/cli/calibre.html

If you want to implement anything beyond the most basic customization you have to implement your own config widget, http://manual.calibre-ebook.com/crea...of-your-plugin

That's what I did, but the config dialog is built by the do_user_config procedure. It puts together my widget and some QDialogButtonBox of its own. I can do anything with my widget, but I don't know how to add buttons to the box that I did not create. I'm guessing the answer is that I can't do that anyway (and it's not a big deal).