[GUI Plugin] Modify ePub

[enuddleyarbl]

I'm experimenting with having the calibre editor convert epub2s to epub3. When I do so and then run Modify ePub on the book, it always seem to find a problem with the cover that doesn't happen with epub2:

The cover image has an id != "cover". Renaming to work around bug in Nook Color

It fixes the issue by duplicating the cover and giving it a new name (cover.jpeg -> cover_u1.jpeg) and doing the same with the cover page (titlepage.xhtml -> titlepage_u1.xhtml). I've got two trivial questions:

1. Why is "_u1" appended to the file names? Is there a meaning there that I'm just missing?
2. Where is that id (that's not equal to "cover") set? I can't seem to find any kind of id in any of those files.

EDIT 1: Also, once Modify ePub creates that new cover/titlepage, running EPUBCheck against the book gives me the following error:

The property "svg" should be declared in the OPF file.

and I have to add:

properties="svg"

to the text/titlepage_u1.xhtml line of content.opf file. With the Semantic cover page set, the whole line looks like:

}</meta>
  </metadata>
  <manifest>
    <item href="text/titlepage_u1.xhtml" id="titlepage1" media-type="application/xhtml+xml" properties="svg calibre:title-page"/>

EDIT 2: Ooh. And running Modify ePub again removes the "properties" bit from the content.opf entirely. I'll have to turn off whatever's doing that in the plugin.

Just for grins, here's the whole Modify ePub log:

Modify ePubs version (1, 7, 3)
Logfile for book ID 1239 (The Abolition of Man / CS Lewis)
Job details 1239
  Modifying:  C:\Users\Dave\AppData\Local\Temp\calibre_smt9rh27\hslu2xep_modify_epub\1239.epub
	Updating metadata and cover
	  Parsing xml file: content.opf
	Looking for redundant entries in manifest
	Looking for unmanifested files
	Looking for broken links in the NCX
	  No NCX found
	Looking for Adobe xpgt files and links to remove
	Looking for pagemaps
	Looking for Google Play pagemaps
	Looking for Adobe DRM meta tags to remove
	Looking for legacy jackets
	Adding or updating jacket
	 Current jacket removed:  jacket.xhtml
	  Manifest item removed: jacket.xhtml (calibre_jacket)
	  Spine itemref removed: calibre_jacket
	  Manifest item added: jacket.xhtml (calibre_jacket)
	  Spine item inserted: calibre_jacket at pos: 1
	Looking for html pages containing only broken image links
	Looking for non dc: elements in manifest
	  Removing child: {http://www.idpf.org/2007/opf}meta
	  Removing child: {http://www.idpf.org/2007/opf}meta
	  Removing child: {http://www.idpf.org/2007/opf}meta
	  Removing child: {http://www.idpf.org/2007/opf}meta
	  Removing child: {http://www.idpf.org/2007/opf}meta
	  Removing child: {http://www.idpf.org/2007/opf}meta
	  Removing child: {http://www.idpf.org/2007/opf}meta
	  Removing child: {http://www.idpf.org/2007/opf}meta
	  Removing child: {http://www.idpf.org/2007/opf}meta
	  Removing child: {http://www.idpf.org/2007/opf}meta
	Looking for html files to remove charset meta tags/encode to utf-8
	  Switched to UTF-8 encoding for: nav.xhtml
	  Switched to UTF-8 encoding for: toc.xhtml
	  Switched to UTF-8 encoding for: text/aboutauthor.html
	  Switched to UTF-8 encoding for: text/aboutpublisher.html
	  Switched to UTF-8 encoding for: text/appendix.html
	  Switched to UTF-8 encoding for: text/booksby.html
	  Switched to UTF-8 encoding for: text/copyrightpage.html
	  Switched to UTF-8 encoding for: text/credits.html
	  Switched to UTF-8 encoding for: text/epigraph.html
	  Switched to UTF-8 encoding for: text/notes.html
	  Switched to UTF-8 encoding for: text/part0004_split_001.html
	  Switched to UTF-8 encoding for: text/part0005_split_001.html
	  Switched to UTF-8 encoding for: text/part0006.html
	  Switched to UTF-8 encoding for: text/tauthorpage.html
	  Switched to UTF-8 encoding for: text/titlepage.xhtml
	  Switched to UTF-8 encoding for: text/tpage.html
	Looking for embedded fonts
	Looking for css @font-face style declarations
	Looking for inline @font-face style declarations
	Looking for CSS margins
	Looking for inline javascript blocks to remove
	Looking for .js files to remove
	Applying smarten punctuation
	Stripping Kobo remnants
	  Stripped Kobo spans in: text/booksby.html
	  Stripped Kobo spans in: text/notes.html
	  Stripped Kobo spans in: text/part0004_split_001.html
	  Stripped Kobo spans in: text/part0005_split_001.html
	  Stripped Kobo spans in: text/part0006.html
	Looking for files to remove: ['iTunesMetadata.plist', 'iTunesArtwork']
	Looking for files to remove: ['META-INF/calibre_bookmarks.txt']
	Looking for files to remove: ['.DS_Store', 'thumbs.db']
	Looking for unused images
	Stripping spans
	Unprettying files
	  De-indented: nav.xhtml
	  De-indented: text/titlepage.xhtml
	Insert or replace cover
	...Looking for guide cover reference
	...Looking for meta cover
	...Looking for other.ms-* guide references to clean out
	...Writing new cover image and titlepage html
	  New cover image written to: images/cover_u1.jpeg
	  New titlepage html written to: text/titlepage_u1.xhtml
	...Updating manifest and TOC for the new cover
	  Manifest item added: text/titlepage_u1.xhtml (titlepage1)
	  Manifest item added: images/cover_u1.jpeg (cover1)
	  Meta item inserted: cover:cover1
	  Spine item inserted: titlepage1 at pos: 0
	  No guide parent element found - inserting one
	  Guide item inserted: text/titlepage_u1.xhtml:Cover:cover
	...Remove any orphaned guide images
	...Apply device specific fixes
The cover image has an id != "cover". Renaming to work around bug in Nook Color
	...Cover modifications completed
ePub updated in 0.36 seconds

[enuddleyarbl]

Trying to stop the plugin from removing those properties, I turned off all the options under Manifest and Metadata. It's still removing them. Any ideas on which option would be doing that?

EDIT: Found it. It's the Covers > Insert or Replace Cover option.

[CheeseJam]

Is there a way to automatically run this plugin when epubs are either imported into Calibre or every time an epub is emailed to a Kindle in Calibre? With Kindle's latest requirement of epub instead of mobi for emailing books, I am getting a lot of weird errors that this plugin seems to be able to fix (like encoding the HTML in UTF-8).

[JSWolf]

Originally Posted by CheeseJam

Is there a way to automatically run this plugin when epubs are either imported into Calibre or every time an epub is emailed to a Kindle in Calibre? With Kindle's latest requirement of epub instead of mobi for emailing books, I am getting a lot of weird errors that this plugin seems to be able to fix (like encoding the HTML in UTF-8).

I don't know if this will work or not, but if this can be done, it would be done using the Action Chains plugin.
https://www.mobileread.com/forums/sh...d.php?t=334974

[CheeseJam]

Originally Posted by JSWolf

I don't know if this will work or not, but if this can be done, it would be done using the Action Chains plugin.
https://www.mobileread.com/forums/sh...d.php?t=334974

I will give that a shot and report back if I have any success--thanks!

[dunhill]

I have translated the plugin into Spanish, I leave a screenshot for some terms that I do not master to see if the translation is correct

[theducks]

Originally Posted by dunhill

I have translated the plugin into Spanish, I leave a screenshot for some terms that I do not master to see if the translation is correct

Is Help supposed to be translated?

[dunhill]

Originally Posted by theducks

Is Help supposed to be translated?

That is why I am asking
In case I have overlooked something or misplaced a term.
Thanks for the indication.

[listyraesder]

calibre, version 6.3.0
ERROR: Modify ePub changed no files: No ePub files were updated. If this isn't what you expected then press the Show details button to check for errors in the log.

Modify ePubs version (1, 7, 3)
Logfile for book ID 7598 (Stephen / Carl Watkins)
Job details 7598
  Modifying:  /tmp/calibre_6.3.0_tmp_izs4c261/jjbi1ux2_modify_epub/7598.epub
	  Parsing xml file: OPS/package.opf
	  Parsing xml file: OPS/toc.ncx
	Looking for broken links in the NCX
	ncx name:  OPS/toc.ncx
	Looping over ncx entries
	Looking for Adobe xpgt files and links to remove
Stephen - ERROR: Traceback (most recent call last):
  File "calibre_plugins.modify_epub.modify", line 78, in process_book
    is_modified = self._process_book(container, options)
  File "calibre_plugins.modify_epub.modify", line 140, in _process_book
    is_changed |= self._remove_xpgt_files(container)
  File "calibre_plugins.modify_epub.modify", line 392, in _remove_xpgt_files
    for match in RE_CSS_IMPORT1.finditer(data):
TypeError: cannot use a string pattern on a bytes-like object

ePub not changed after 0.32 seconds

So this is occurring on a small handful of books.

• The plugin runs fine with Remove xpgt files unchecked.
• The files are all EPUB 3.
• The files don't have xpgt files inside them.

[JSWolf]

Originally Posted by listyraesder

calibre, version 6.3.0
ERROR: Modify ePub changed no files: No ePub files were updated. If this isn't what you expected then press the Show details button to check for errors in the log.

Modify ePubs version (1, 7, 3)
Logfile for book ID 7598 (Stephen / Carl Watkins)
Job details 7598
  Modifying:  /tmp/calibre_6.3.0_tmp_izs4c261/jjbi1ux2_modify_epub/7598.epub
	  Parsing xml file: OPS/package.opf
	  Parsing xml file: OPS/toc.ncx
	Looking for broken links in the NCX
	ncx name:  OPS/toc.ncx
	Looping over ncx entries
	Looking for Adobe xpgt files and links to remove
Stephen - ERROR: Traceback (most recent call last):
  File "calibre_plugins.modify_epub.modify", line 78, in process_book
    is_modified = self._process_book(container, options)
  File "calibre_plugins.modify_epub.modify", line 140, in _process_book
    is_changed |= self._remove_xpgt_files(container)
  File "calibre_plugins.modify_epub.modify", line 392, in _remove_xpgt_files
    for match in RE_CSS_IMPORT1.finditer(data):
TypeError: cannot use a string pattern on a bytes-like object

ePub not changed after 0.32 seconds

So this is occurring on a small handful of books.

• The plugin runs fine with Remove xpgt files unchecked.
• The files are all EPUB 3.
• The files don't have xpgt files inside them.

What other options do you have checked? I do have Remove Adobe .xpgt files and links checked and I do not have any error with Calibre 6.3.0.

[theducks]

Jon
I think the point is having no XPGT file , should just fail/skip gracefully as it does in EPUB2 (I have that checked, but I now use Polish more the Modify, so I may not have triggered this error)

[JSWolf]

Originally Posted by theducks

Jon
I think the point is having no XPGT file , should just fail/skip gracefully as it does in EPUB2 (I have that checked, but I now use Polish more the Modify, so I may not have triggered this error)

ePub these days do not have an XPGT file. So when I've been using this plugin, I've had no XPGT with ePub 2 & ePub 3.

I don't get any errors. So knowing what other options are checked may help pinpoint the problem.

[davidfor]

Originally Posted by listyraesder

calibre, version 6.3.0
ERROR: Modify ePub changed no files: No ePub files were updated. If this isn't what you expected then press the Show details button to check for errors in the log.

Modify ePubs version (1, 7, 3)
Logfile for book ID 7598 (Stephen / Carl Watkins)
Job details 7598
  Modifying:  /tmp/calibre_6.3.0_tmp_izs4c261/jjbi1ux2_modify_epub/7598.epub
	  Parsing xml file: OPS/package.opf
	  Parsing xml file: OPS/toc.ncx
	Looking for broken links in the NCX
	ncx name:  OPS/toc.ncx
	Looping over ncx entries
	Looking for Adobe xpgt files and links to remove
Stephen - ERROR: Traceback (most recent call last):
  File "calibre_plugins.modify_epub.modify", line 78, in process_book
    is_modified = self._process_book(container, options)
  File "calibre_plugins.modify_epub.modify", line 140, in _process_book
    is_changed |= self._remove_xpgt_files(container)
  File "calibre_plugins.modify_epub.modify", line 392, in _remove_xpgt_files
    for match in RE_CSS_IMPORT1.finditer(data):
TypeError: cannot use a string pattern on a bytes-like object

ePub not changed after 0.32 seconds

So this is occurring on a small handful of books.

• The plugin runs fine with Remove xpgt files unchecked.
• The files are all EPUB 3.
• The files don't have xpgt files inside them.

The line in error is using a regex to search for the import of the XPGT files in the CSS and HTMLX files. The error is that the text pulled from these is not a string, but, a bytes-like object. That suggests that the file causing the error has the wrong encoding. It's possible that the error check in the calibre editor will show which file has the error. Otherwise, we would have to see the book to see why it is going wrong.

[listyraesder]

The only things error check flags are !important CSS rules.

[davidfor]

Originally Posted by listyraesder

The only things error check flags are !important CSS rules.

I had a look at this today, and I cannot reproduce the problem. When I run the plugin with the XPGT options, it runs with no errors and no changes. Running with the two options that I can see in your output, it works and no changes are made.

One thing that the error does not tell me is which file is the problem. I have attached a test version that prints the name of the file and the raw contents to the log. That might give a clue. Could your try this and post the log?

The only other thing I can think of is language or locale settings. I don't think they should have an affect, but, there have been some other issues.

My testing is on Windows10. Your log looks like either you are using either Mac or Linux. This shouldn't affect anything, but...