[GUI Plugin] Modify ePub

[chaley]

Originally Posted by hobnail

See if this helps: https://docs.python.org/3/howto/pypo...us-binary-data

The problem isn't knowing what is required. It is instead finding all the places where it matters. For example, it wasn't obvious that the common read method is sometimes used for images. As the referred-to document says

(it is highly recommended you don’t design APIs that can take both due to the difficulty of keeping the code working; as stated earlier it is difficult to do well).

Other than rewriting the plugin not to use a common interface, something I am not willing to do, the fix is to check the file extension and use 'rb' if it is an image extension and 'r' otherwise.

EDIT: Sorry if I sound a bit snippy. I do appreciate the suggestions.

[DNSB]

Originally Posted by chaley

Yet another test version. This one corrects problems caused image files.

On my computer, version 1.5.4 abended in container.py line 14.

Spoiler:

calibre, version 5.0.1 (win32, embedded-python: True)
Failed to modify ePubs: Failed: Modify ePubs

Modify ePubs
Traceback (most recent call last):
  File "runpy.py", line 194, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "site.py", line 89, in <module>
  File "site.py", line 84, in main
  File "site.py", line 55, in run_entry_point
  File "site-packages\calibre\utils\ipc\worker.py", line 215, in main
  File "site-packages\calibre\utils\ipc\worker.py", line 143, in arbitrary_n
  File "importlib\__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "site-packages\calibre\customize\zipplugin.py", line 184, in load_module
  File "calibre_plugins.modify_epub.jobs", line 14, in <module>
  File "site-packages\calibre\customize\zipplugin.py", line 184, in load_module
  File "calibre_plugins.modify_epub.modify", line 22, in <module>
  File "site-packages\calibre\customize\zipplugin.py", line 184, in load_module
  File "calibre_plugins.modify_epub.container", line 14, in <module>
ModuleNotFoundError: No module named 'urlparse'

See attached image for the options used though I suspect it's not getting that far.

[JSWolf]

Originally Posted by hobnail

That would merely obscure/hide the underlying bug in the plugin's code, not fix it.

I know it's not a fix. But it is a temporary fix until a proper fix is in place.

[chaley]

Originally Posted by DNSB

On my computer, version 1.5.4 abended in container.py line 14.

Spoiler:

calibre, version 5.0.1 (win32, embedded-python: True)
Failed to modify ePubs: Failed: Modify ePubs

Modify ePubs
Traceback (most recent call last):
  File "runpy.py", line 194, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "site.py", line 89, in <module>
  File "site.py", line 84, in main
  File "site.py", line 55, in run_entry_point
  File "site-packages\calibre\utils\ipc\worker.py", line 215, in main
  File "site-packages\calibre\utils\ipc\worker.py", line 143, in arbitrary_n
  File "importlib\__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "site-packages\calibre\customize\zipplugin.py", line 184, in load_module
  File "calibre_plugins.modify_epub.jobs", line 14, in <module>
  File "site-packages\calibre\customize\zipplugin.py", line 184, in load_module
  File "calibre_plugins.modify_epub.modify", line 22, in <module>
  File "site-packages\calibre\customize\zipplugin.py", line 184, in load_module
  File "calibre_plugins.modify_epub.container", line 14, in <module>
ModuleNotFoundError: No module named 'urlparse'

See attached image for the options used though I suspect it's not getting that far.

That is very strange. It is saying that the plugin can't import a standard calibre module, in this case six.moves.urllib.request. What OS are you using?

EDIT: I just tried it using stock calibre 5.0.1 (not running from source) on Win10 Pro and had no problem.

[JSWolf]

I cannot get 1.5.4 to error with 4.23. I'll try 5.0.1 portable.

[DNSB]

Originally Posted by chaley

That is very strange. It is saying that the plugin can't import a standard calibre module, in this case six.moves.urllib.request. What OS are you using?

I looked at the 1.5.4 container.py file and lines 10 to 15 read:

import os, posixpath, sys, re, urllib

from lxml import etree
from lxml.etree import XMLSyntaxError
from urlparse import urldefrag, urlparse, urlunparse
from urllib import unquote as urlunquote

while the 1.5.3 container.py file has:

import six
from six.moves import range
from polyglot.builtins import unicode_type

__license__   = 'GPL v3'
__copyright__ = '2011, Grant Drake <grant.drake@gmail.com>'
__docformat__ = 'restructuredtext en'

import os, posixpath, sys, re
import six.moves.urllib.request, six.moves.urllib.parse, six.moves.urllib.error
from io import open

from lxml import etree
from lxml.etree import XMLSyntaxError
from six.moves.urllib.parse import urldefrag, urlparse, urlunparse
from six.moves.urllib.parse import unquote as urlunquote

I copied the segment from the 1.5.3 container.py into the 1.5.4 container.py and now I get a different error message. I tried 4 or 5 single options and the same error.

Spoiler:

calibre, version 5.0.1
ERROR: Modify ePub failed: No ePub files were updated

Modify ePubs

  Modifying:  C:\Users\David\AppData\Local\Temp\calibre_y3si84vr\81p9w4o7_modify_epub\9905.epub

Dealing With Demons - ERROR: Traceback (most recent call last):

  File "calibre_plugins.modify_epub.modify", line 60, in process_book

  File "calibre_plugins.modify_epub.modify", line 942, in _setup_user_options

  File "calibre_plugins.modify_epub.modify", line 921, in get_user_margins

AttributeError: 'GuiRecommendations' object has no attribute 'iteritems'



ePub not changed after 0.04 seconds

This is with calibre 5.0.1, Windows 10 x64.

[JSWolf]

Originally Posted by JSWolf

I cannot get 1.5.4 to error with 4.23. I'll try 5.0.1 portable.

Trying 1.5.4 with Calibre 5.0.1, I get a new error. I clicked Save to save my settings and got...

calibre, version 5.0.1
ERROR: Unhandled exception: <b>AttributeError</b>:'dict' object has no attribute 'iteritems'

calibre 5.0.1 Portable embedded-python: True is64bit: False
Windows-10-10.0.18362-SP0 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.18362')
Python 3.8.5
Windows: ('10', '10.0.18362', 'SP0', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Amazon hi-res covers (0, 1, 0) && Annotations (1, 12, 0) && BookFusion Plugin (0, 5, 3) && Count Pages (1, 10, 0) && Diaps Editing Toolbag (0, 3, 6) && EpubCheck (0, 2, 3) && EpubSplit (2, 9, 0) && Favourites Menu (1, 0, 5) && Find Duplicates (1, 7, 2) && Hyphenate This! (0, 0, 9) && Set KFX metadata (from KFX Output) (1, 44, 0) && KFX Output (1, 44, 0) && KePub Metadata Reader (3, 2, 9) && KePub Metadata Writer (3, 2, 12) && KePub Output (3, 2, 15) && KindleUnpack - The Plugin (0, 82, 3) && Kobo Books (1, 8, 0) && Kobo Utilities (2, 11, 5) && KoboTouchExtended (3, 2, 15) && Modify ePub (1, 4, 1) && Quality Check (1, 10, 1) && Reading List (1, 6, 15) && ScrambleEbook (0, 4, 5)
Traceback (most recent call last):
  File "calibre_plugins.modify_epub.dialogs", line 198, in _save_clicked
AttributeError: 'dict' object has no attribute 'iteritems'

[chaley]

Folks, I think i messed up and posted the original unported version. I will put up the new one after I finish dinner.

[JSWolf]

Originally Posted by chaley

Folks, I think i messed up and posted the original unported version. I will put up the new one after I finish dinner.

I'll test again with both 4.23 and 5.0.1 when after your dinner.

[DNSB]

Originally Posted by chaley

Folks, I think i messed up and posted the original unported version. I will put up the new one after I finish dinner.

Thanks. I really appreciate the effort you are going to to make Modify Epub work with calibre 5.

[JSWolf]

I too appreciate this. I really do hope we can get it working. It's one of the plugins I use a lot.

[chaley]

Here is the latest version -- V1.5.4. This time I checked.

[DNSB]

Still no joy here. I tried 4 or 5 individual options and the error message was the same.

calibre, version 5.0.1
ERROR: Modify ePub failed: No ePub files were updated

Modify ePubs

  Modifying:  C:\Users\David\AppData\Local\Temp\calibre_dj7nyszb\rw5zcx9r_modify_epub\9904.epub

	Updating metadata and cover

Mind Games - ERROR: Traceback (most recent call last):

  File "calibre_plugins.modify_epub.modify", line 78, in process_book

  File "calibre_plugins.modify_epub.container", line 86, in __init__

  File "src/lxml/etree.pyx", line 3237, in lxml.etree.fromstring

  File "src/lxml/parser.pxi", line 1891, in lxml.etree._parseMemoryDocument

ValueError: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.


ePub not changed after 0.30 seconds

If I modify line 86 in container.py to change 'r' to 'rb', I don't get the error message but ghod alone knows what else that is breaking.

[JSWolf]

Originally Posted by chaley

Here is the latest version -- V1.5.4. This time I checked.

Still not working,

Calibre 4.23 any option..

calibre, version 4.23.0
ERROR: Modify ePub failed: No ePub files were updated

Modify ePubs
Logfile for book ID 2244 (The Measure of the Magic / Terry Brooks)
2244
  Modifying:  C:\Users\help\AppData\Local\Temp\calibre_xn2tdk\ggfek8_modify_epub\2244.epub
The Measure of the Magic - ERROR: Traceback (most recent call last):
  File "calibre_plugins.modify_epub.modify", line 78, in process_book
  File "calibre_plugins.modify_epub.container", line 86, in __init__
  File "src/lxml/etree.pyx", line 3222, in lxml.etree.fromstring
  File "src/lxml/parser.pxi", line 1872, in lxml.etree._parseMemoryDocument
ValueError: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.

ePub not changed after 0.89 seconds

Calibre 5.0.1 any option...

calibre, version 5.0.1
ERROR: Modify ePub failed: No ePub files were updated

Modify ePubs
Logfile for book ID 2244 (The Measure of the Magic / Terry Brooks)
2244

  Modifying:  C:\Users\help\AppData\Local\Temp\calibre_179jmqy8\i1y3qr4f_modify_epub\2244.epub

The Measure of the Magic - ERROR: Traceback (most recent call last):

  File "calibre_plugins.modify_epub.modify", line 78, in process_book

  File "calibre_plugins.modify_epub.container", line 86, in __init__

  File "src/lxml/etree.pyx", line 3237, in lxml.etree.fromstring

  File "src/lxml/parser.pxi", line 1891, in lxml.etree._parseMemoryDocument

ValueError: Unicode strings with encoding declaration are not supported. Please use bytes input or XML fragments without declaration.



ePub not changed after 0.72 seconds

[chaley]

@DNSB: thanks for the exception log, and @JSWolf: thanks for the copy of an offending book. That let me find another case where something the plugin calls sometimes wants a byte string instead of a "normal" string. The attached version takes care of that.

I tested it with your book that specifies the encoding, and with one of my books that does not. I checked every box. No fatal failures. The plugin did complain about lacking a cover in your book.

May it please be that this one works.