[GUI Plugin] Modify ePub

[jackie_w]

Originally Posted by DNSB

1. AFAIR, calibre (Polish, Save to, Send to or Modify Epub) will modify the metadata but not the manifest or spine segments of the .opf file internal to the epub. If I edit the epub, all bets are off.

Perhaps I should make it clear that I would like this option when Modify Epub is inserting or replacing a cover and the epub version is 3.

titlepage.xhtml ERROR(OPF-014):  The property 'svg' should be declared in the OPF file.

I think calibre's builtin Polish Update Cover option does make any necessary OPF changes for epub3.

[DNSB]

Originally Posted by Rev. Bob

Next up: Can you point me at the specific sections of the EPUB 3 specs (I don’t know whether the requirement has changed in any of the point versions) which discuss this, so I know exactly what is called for and precisely where?

The latest version (I think) shows in the E.2 Manifest item Properties in section E.2.6.

A minor nit but if it can be fixed, would be much appreciated.

Basically, I need a line which reads:

<item id="titlepage" href="xhtml/titlepage.xhtml" media-type="application/xhtml+xml"/>

modified to:

<item id="titlepage" href="xhtml/titlepage.xhtml" media-type="application/xhtml+xml" properties="svg"/>

[DNSB]

Originally Posted by jackie_w

I think calibre's builtin Polish Update Cover option does make any necessary OPF changes for epub3.

That and more as it adds properties="calibre:title-page svg".

I prefer Modify Epub as it does more for me than Polish does--I often remove unwanted spans by simply editing the <span class="blahblah"> to <span>, etc. and then let Modify Epub clean up the remnants.

One common reason for this is the body sets a default font size of 1.0em, the paragraph sets a default size of 1.0em and then each covered in red body fluid paragraph has a span with a class that sets font size to 1.0em.

[Rev. Bob]

Originally Posted by DNSB

The latest version (I think) shows in the E.2 Manifest item Properties in section E.2.6.

A minor nit but if it can be fixed, would be much appreciated.

Not so minor, to do it properly:

This property MUST be set when SVG markup is included directly in the resource and MAY be set when the SVG is referenced from the resource (e.g., from an [HTML] img, object or iframe element).

In other words, while the cover document is the most common place this would be found, it’s not at all the only one. A proper implementation of this feature would have to scan every file in the ebook for SVG markup and flag all of them appropriately. That’s a significantly nastier can of worms.

I mean, there’s nothing conceptually wrong with saying, in effect, “I’m changing the cover and specifying SVG markup, therefore I should check the EPUB version and see if I need to slap the properties attribute on it in the OPF” – but that still leaves the rest of the ebook unchecked, just waiting to generate those validation errors you want to avoid.

And then, on top of all that, the code has to deal with the possibility that a properties attribute already exists, meaning it has to be checked and correctly modified…

[DNSB]

Originally Posted by Rev. Bob

Not so minor, to do it properly:

In other words, while the cover document is the most common place this would be found, it’s not at all the only one. A proper implementation of this feature would have to scan every file in the ebook for SVG markup and flag all of them appropriately. That’s a significantly nastier can of worms.

I was not asking for Modify Epub to check every file and add that propery if needed. What I do want is for Modify Epub to do this for the cover file it inserts. The other files are either going to be already correctly labelled or I will have fixed them as part of my cleanup before I run Modify Epub.

Originally Posted by Rev. Bob

I mean, there’s nothing conceptually wrong with saying, in effect, “I’m changing the cover and specifying SVG markup, therefore I should check the EPUB version and see if I need to slap the properties attribute on it in the OPF” – but that still leaves the rest of the ebook unchecked, just waiting to generate those validation errors you want to avoid.

And then, on top of all that, the code has to deal with the possibility that a properties attribute already exists, meaning it has to be checked and correctly modified…

Again, I wasn't asking for Modify Epub to check this for the entire epub, just for a cover image inserted into an epub3 by Modify Epub.

[BetterRed]

Just a reminder that the people maintaining kiwidude's plugins, are doing us a significant favour - voluntarily. Life support maintenance, i.e. ensuring a plugin continues to work on calibre's baseline must be the first priority.

If the original developer of a plugin declines to implement a functional change then so be it, same applies to maintainers of 'abandoned' plugins only more especially so. The plugin source is in the zip, you know the rest…

BR

[Rev. Bob]

Originally Posted by DNSB

I was not asking for Modify Epub to check every file and add that propery if needed. What I do want is for Modify Epub to do this for the cover file it inserts. The other files are either going to be already correctly labelled or I will have fixed them as part of my cleanup before I run Modify Epub.

Thing is, that’s your expectation given your specific situation. I get that.

But I also know that other users will see that the plugin addresses Case X and wonder why it doesn’t also handle the very similar Cases Y and Z, and I have to consider that.

I’m not saying no. I just haven’t decided to say yes.

[DNSB]

Originally Posted by Rev. Bob

Thing is, that’s your expectation given your specific situation. I get that.

But I also know that other users will see that the plugin addresses Case X and wonder why it doesn’t also handle the very similar Cases Y and Z, and I have to consider that.

I’m not saying no. I just haven’t decided to say yes.

Sadly true. Hopefully, you will say yes but if the answer is no, I can understand why.

[icallaci]

+1. If it's worth anything, I would also like properties="svg" inserted into the opf when a cover is added to an epub3. Right now I do it manually, but it would be really nice if the plugin took care of this and saved me an extra step. However, I also understand the concept of slippery slopes, so I understand if the answer is no.

[JSWolf]

As long as were asking for fixes...

There is a bug where graphics get deleted if they are used in the CSS since Modify ePub is not checking the CSS. Can this please be fixed? Some eBooks specify graphics in the CSS for use as a section break marker. But Modify ePub deletes this since it thinks the graphic is unused. If Modify ePub checked the CSS for graphics, then this bug can be put to rest.

[Rev. Bob]

Originally Posted by JSWolf

As long as were asking for fixes...

There is a bug where graphics get deleted if they are used in the CSS since Modify ePub is not checking the CSS. Can this please be fixed?

Feel free to write the code for it. I’ll be glad to take a look and consider the patch once you have a stable implementation.

[JSWolf]

Originally Posted by Rev. Bob

Feel free to write the code for it. I’ll be glad to take a look and consider the patch once you have a stable implementation.

I don't know Python. So that's out.

[salfter]

The plugin linked in the OP appears to not be compatible with Python 3. With Python 2 having effectively been EOL'd this year, a version that is compatible with Python 3 would be useful.

Here's my attempt at such a conversion:

https://home.alfter.us/s/aQcprygAZLSpd2B

I've bumped the version number to 1.4.2, set the minimum Calibre version to 4.0, and kept running the converter against an ePub until it ran without errors. These are my changes:

diff -ur "Modify ePub.py2/common_utils.py" "Modify ePub.py3/common_utils.py"
--- "Modify ePub.py2/common_utils.py"   2015-01-11 10:33:06.000000000 -0800
+++ "Modify ePub.py3/common_utils.py"   2020-08-15 12:49:43.422848052 -0700
@@ -438,7 +438,7 @@
     def populate_combo(self, selected_key):
         self.clear()
         selected_idx = idx = -1
-        for key, value in self.values.iteritems():
+        for key, value in self.values.items():
             idx = idx + 1
             self.addItem(value)
             if key == selected_key:
@@ -446,7 +446,7 @@
         self.setCurrentIndex(selected_idx)
 
     def selected_key(self):
-        for key, value in self.values.iteritems():
+        for key, value in self.values.items():
             if value == unicode(self.currentText()).strip():
                 return key
 
diff -ur "Modify ePub.py2/container.py" "Modify ePub.py3/container.py"
--- "Modify ePub.py2/container.py"      2020-03-03 18:40:15.000000000 -0800
+++ "Modify ePub.py3/container.py"      2020-08-15 13:31:05.561700105 -0700
@@ -11,8 +11,8 @@
 
 from lxml import etree
 from lxml.etree import XMLSyntaxError
-from urlparse import urldefrag, urlparse, urlunparse
-from urllib import unquote as urlunquote
+from urllib.parse import urldefrag, urlparse, urlunparse
+from urllib.parse import unquote as urlunquote
 
 from calibre import guess_type, prepare_string_for_xml
 from calibre.ebooks.chardet import xml_to_unicode
@@ -170,11 +170,11 @@
         if not base and rel_to_opf:
             base = self.opf_dir
         if not base:
-            return urllib.quote(name)
+            return urllib.parse.quote(name)
         href = posixpath.relpath(name, base)
         if href == '.':
             href = ''
-        return urllib.quote(href)
+        return urllib.parse.quote(href)
 
     def abshref(self, href, base_name):
         """Convert the URL provided in :param:`href` from a reference
@@ -303,7 +303,7 @@
         """Automatically decode :param:`data` into a `unicode` object."""
         def fix_data(d):
             return d.replace('\r\n', '\n').replace('\r', '\n')
-        if isinstance(data, unicode):
+        if isinstance(data, str):
             return fix_data(data)
         bom_enc = None
         if data[:4] in ('\0\0\xfe\xff', '\xff\xfe\0\0'):
@@ -682,7 +682,7 @@
         self.dirtied.clear()
         with ZipFile(path, 'w', compression=ZIP_DEFLATED) as zf:
             # Write mimetype
-            zf.writestr('mimetype', bytes(guess_type('a.epub')[0]),
+            zf.writestr('mimetype', bytes(guess_type('a.epub')[0], "utf-8"),
                     compression=ZIP_STORED)
             # Write everything else
             exclude_files = ['.DS_Store','mimetype']
@@ -863,7 +863,7 @@
         data = self.get_parsed_etree(html_name)
         body = XPath('//h:body')(data)
         if body:
-            text = etree.tostring(body[0], method='text', encoding=unicode)
+            text = etree.tostring(body[0]).decode()
         else:
             text = ''
         text = re.sub(r'\s+', '', text)
diff -ur "Modify ePub.py2/covers.py" "Modify ePub.py3/covers.py"
--- "Modify ePub.py2/covers.py" 2012-07-03 09:11:08.000000000 -0700
+++ "Modify ePub.py3/covers.py" 2020-08-15 13:28:40.400708757 -0700
@@ -8,7 +8,8 @@
 __docformat__ = 'restructuredtext en'
 
 import posixpath, os
-from urllib import unquote
+from urllib.parse import unquote
+from functools import cmp_to_key
 
 from calibre import fit_image
 from calibre.utils.magick.draw import Image
@@ -229,7 +230,7 @@
                 self.container.delete_from_guide(reference)
 
         # Sort by the largest size
-        covers.sort(cmp=lambda x,y:cmp(x[1], y[1]), reverse=True)
+        covers.sort(key=cmp_to_key(lambda x,y:cmp(x[1], y[1])), reverse=True)
         if covers:
             reference = covers[0][0]
             href = reference.get('href')
@@ -367,7 +368,7 @@
         tp = templ%unquote(rel_cover_href)
 
         with open(titlepage_path, 'wb') as f:
-            f.write(tp)
+            f.write(bytes(tp, "utf-8"))
         titlepage_name = os.path.relpath(titlepage_path, self.container.root).replace(os.sep, '/')
         return titlepage_name
 
diff -ur "Modify ePub.py2/css.py" "Modify ePub.py3/css.py"
--- "Modify ePub.py2/css.py"    2013-09-01 00:59:38.000000000 -0700
+++ "Modify ePub.py3/css.py"    2020-08-15 12:49:43.423848052 -0700
@@ -27,7 +27,7 @@
         from calibre.ebooks.conversion.config import load_defaults
         ps = load_defaults('look_and_feel')
         # Only interested in the extra_css out of settings
-        prefs_css = dict((k,v) for k,v in ps.iteritems() if k == 'extra_css')
+        prefs_css = dict((k,v) for k,v in ps.items() if k == 'extra_css')
         return prefs_css.get('extra_css', '')
 
     def rewrite_css(self):
diff -ur "Modify ePub.py2/dialogs.py" "Modify ePub.py3/dialogs.py"
--- "Modify ePub.py2/dialogs.py"        2019-10-17 03:14:00.000000000 -0700
+++ "Modify ePub.py3/dialogs.py"        2020-08-15 12:49:43.424848051 -0700
@@ -195,7 +195,7 @@
 
     def _save_clicked(self):
         self._set_options()
-        cfg.plugin_prefs[cfg.STORE_SAVED_SETTINGS] = [k for k,v in self.options.iteritems() if v]
+        cfg.plugin_prefs[cfg.STORE_SAVED_SETTINGS] = [k for k,v in self.options.items() if v]
 
     def _restore_clicked(self):
         self._select_none_clicked()
diff -ur "Modify ePub.py2/__init__.py" "Modify ePub.py3/__init__.py"
--- "Modify ePub.py2/__init__.py"       2020-03-12 16:08:36.000000000 -0700
+++ "Modify ePub.py3/__init__.py"       2020-08-15 12:45:15.147864042 -0700
@@ -24,8 +24,8 @@
     description             = 'Apply cleanup tasks and updates to an ePub without doing a conversion'
     supported_platforms     = ['windows', 'osx', 'linux']
     author                  = 'Grant Drake, with additions by Robert L. Hood and Leigh Parry'
-    version                 = (1, 4, 1)
-    minimum_calibre_version = (0, 8, 53)
+    version                 = (1, 4, 2)
+    minimum_calibre_version = (4, 0, 0)
 
     #: This field defines the GUI plugin class that contains all the code
     #: that actually does something. Its format is module_path:class_name
diff -ur "Modify ePub.py2/margins.py" "Modify ePub.py3/margins.py"
--- "Modify ePub.py2/margins.py"        2012-12-09 16:25:16.000000000 -0800
+++ "Modify ePub.py3/margins.py"        2020-08-15 12:49:43.424848051 -0700
@@ -33,7 +33,7 @@
         from calibre.ebooks.conversion.config import load_defaults
         ps = load_defaults('page_setup')
         # Only interested in the margins out of page setup settings
-        prefs_margins = dict((k,v) for k,v in ps.iteritems() if k.startswith('margin_'))
+        prefs_margins = dict((k,v) for k,v in ps.items() if k.startswith('margin_'))
         if 'margin_top' not in prefs_margins:
             # The user has never changed their page setup defaults to save settings
             prefs_margins = calibre_default_margins
@@ -41,7 +41,7 @@
 
     def _prefs_to_css_properties(self, user_margins):
         css_margins = ''
-        for pref, value in user_margins.iteritems():
+        for pref, value in user_margins.items():
             # Negative margins mean we don't want the attribute written
             if value >= 0.0:
                 property_name = re.sub('_', '-', pref)
@@ -86,7 +86,7 @@
 
         # If we got to here, then we found "some" margins in the style that are
         # either identical or a subset of our preferred margins
-        for pref, pref_value in self.user_margins.iteritems():
+        for pref, pref_value in self.user_margins.items():
             if pref_value < 0.0:  # The user does not want this margin defined
                 if pref in doc_defined_margins:  # Currently is defined, so remove it
                     return True
diff -ur "Modify ePub.py2/modify.py" "Modify ePub.py3/modify.py"
--- "Modify ePub.py2/modify.py" 2020-03-03 18:38:14.000000000 -0800
+++ "Modify ePub.py3/modify.py" 2020-08-15 12:49:43.425848051 -0700
@@ -918,7 +918,7 @@
             else:
                 prefs_margins = default_margins
 
-            for s, v in prefs_margins.iteritems():
+            for s, v in prefs_margins.items():
                 setattr(self.opts, s, v)
 
         def get_epub_output_options():
@@ -935,7 +935,7 @@
             else:
                 prefs_options = default_values
 
-            for s, v in prefs_options.iteritems():
+            for s, v in prefs_options.items():
                 setattr(self.opts, s, v)
 
         self.opts = OptionValues()

[Rev. Bob]

If I were conversant in Python 3, I’d be glad to evaluate this… but I’m not.

Any takers?

[capink]

I don't use this plugin myself, but here is a link for a version I sent to @salfter in a pm. It is partly based on his port, but modified to work on both python2 and python3.

I didn't test this one, so it should be treated as a base to build on, rather than a functional complete port.

Edit: Trying out this version, it is obvious it needs a lot of work. This is better done by someone who is familiar with this plugin.