Need help diagnosing a plugin in Python 2

[EbookMakers]

An active group of calibre users are using an editor plugin to polish public domain epubs from OCR according to French typography rules.

The add-in attached is a software package in Python that allows you to choose and launch saved searches from a pack of over a hundred regex-functions. The regex pack is updated regularly based on feedback from users. Flags allow communication between the saved searches package and the software envelope. The plugin could possibly be used in other languages, but with new, adapted saved searches.

Whoever wrote the software suddenly disappeared a few months ago. We have reason to assume, without being sure, that he considered the need for compatibility with Python 3 because he had the skills to do so.

We can't even install the add-in in calibre 5.1. We don't have the Python proficiency. We are looking for a goodwill who would like to make a diagnosis with the tools, in order to have an idea of ​​the extent of the conversion work to be done.

[DaltonST]

I did not install this plugin, but simply doing a basic syntax check in Python 3.8 independently installed on my PC revealed a trivial error occurring at least twice:

Python 2: reg = ur'{}'.format(kfind)

The "ur'" in Python 2 means "unicode raw string". Python 3 is natively unicode.

Search on ur' and change it to r'.

Python 3: reg = r'{}'.format(kfind)



DaltonST

[EbookMakers]

Thanks a lot, DaltonST !

[eschwartz]

In addition to the two ur'string' instances previously mentioned...

You also need to change "from Queue import Queue" to "from queue import Queue" on line 15 of main.py (the module was renamed to lowercase).

In the same file on line 108,

with open(os.path.join(config_dir, 'dictionaries', fn), 'w') as f:

must be

with open(os.path.join(config_dir, 'dictionaries', fn), 'wb') as f:

zipfile.ZipFile('something.zip').read('a_file.txt' ) returns bytes and must therefore be written to f.write() using bytes.

I could not find anything else obviously erroring out, the plugin now loads and provides a menu of options that seem to work. On the other hand, I don't read French so I have no idea what it says, and couldn't test anything other than randomly clicking every button and checking for python runtime errors.

If there are any other errors, post them here and you're sure to find people willing to guide you in debugging it. But here's a tip when you do so: start the ebook editor from the command line, using the command:

calibre-debug -t

This will let you see the python errors, and post them for analysis. e.g. the original plugin:

  File "/home/eschwartz/git/calibre/src/calibre/gui2/tweak_book/plugin.py", line 115, in load_plugin_tools
    main = importlib.import_module(plugin.__class__.__module__+'.main')
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "/home/eschwartz/git/calibre/src/calibre/customize/zipplugin.py", line 179, in load_module
    compiled = compile(code, 'calibre_plugins.%s.%s'%(plugin_name,
  File "calibre_plugins.typex.main", line 328
    reg = ur'{}'.format(kfind)
            ^
SyntaxError: invalid syntax

This pointed to the first error needing to be fixed.
Followed by

  File "/home/eschwartz/git/calibre/src/calibre/customize/zipplugin.py", line 184, in load_module
    exec(compiled, mod.__dict__)
  File "calibre_plugins.typex.main", line 15, in <module>
ModuleNotFoundError: No module named 'Queue'

Then

  File "/home/eschwartz/git/calibre/src/calibre/customize/zipplugin.py", line 179, in load_module
    compiled = compile(code, 'calibre_plugins.%s.%s'%(plugin_name,
  File "calibre_plugins.typex.jobutils", line 82
    reg = ur'{}'.format(kfind)
            ^
SyntaxError: invalid syntax

[EbookMakers]

Many, many thanks, eschwartz, for this comprehensive answer. We will do so, and test, test. We are a little reassured about the extent of migration.

[Doitsu]

@EbookMakers

I converted the plugin with the 2to3 utility and attached it as a starting point for further updates.

It can be installed and some of the features appear to work, but others unfortunately don't.

It's definitely not ready for production use!!!

Someone else who's knows more than me about the Calibre plugin API should definitely double-check my changes, in particular all code that reads and writes container files.

[JSWolf]

Is there any way to run Python-Modernize with Windows 10?

[DNSB]

Originally Posted by JSWolf

Is there any way to run Python-Modernize with Windows 10?

Install Python 3.8.whatever then use pip install modernize. If you have installed Python other than in the default AppData location (I like it to be in Program Files), I'd recommend running that command from a administrative command prompt. Assuming the executables are on your path, you can either run:

python -m modernize -w file_to_be_updated.py

python-modernize file_to_be_updated.py

For help, run python -m modernize --help or python-modernize --help.

Just note that this will not give you a guaranteed to run Python 3 file, just that a lot of the gruntwork will have been done.

[EbookMakers]

Thank you Doitsu, and all of you. You bring us the material to start working. We're getting started! All our gratitude!

[JSWolf]

Originally Posted by DNSB

Install Python 3.8.whatever then use pip install modernize. If you have installed Python other than in the default AppData location (I like it to be in Program Files), I'd recommend running that command from a administrative command prompt. Assuming the executables are on your path, you can either run:

python -m modernize -w file_to_be_updated.py

python-modernize file_to_be_updated.py

For help, run python -m modernize --help or python-modernize --help.

Just note that this will not give you a guaranteed to run Python 3 file, just that a lot of the gruntwork will have been done.

Thank you. I'll give it a go when I have the time. I'm really curious how it would deal with the DeDRM plugin.

[DNSB]

Originally Posted by JSWolf

Thank you. I'll give it a go when I have the time. I'm really curious how it would deal with the DeDRM plugin.

Not very well and the output is intended for Python 3. Adding back in Python 2 compatibility and fixing any leftover errors is not fun. Much like what @chaley went through with the ModifyEpub plugin.

[jhowell]

Originally Posted by DNSB

Not very well and the output is intended for Python 3. Adding back in Python 2 compatibility and fixing any leftover errors is not fun. Much like what @chaley went through with the ModifyEpub plugin.

From what I can see no attempt is being made to make the next version of DeDRM backward compatible with Python 2. It will only be usable in calibre 5 or later.

[JSWolf]

Originally Posted by DNSB

Not very well and the output is intended for Python 3. Adding back in Python 2 compatibility and fixing any leftover errors is not fun. Much like what @chaley went through with the ModifyEpub plugin.

Given that DeDRM is not distributed with the Calibre plugin system, I see no reason to to have 6.8.0 for 4.23 or earlier and 7.0.1 for 5.x. So if it was to work for some reason, then we'd have two versions.

[DNSB]

Originally Posted by JSWolf

Given that DeDRM is not distributed with the Calibre plugin system, I see no reason to to have 6.8.0 for 4.23 or earlier and 7.0.1 for 5.x. So if it was to work for some reason, then we'd have two versions.

Much like the discussion in the Modify Epub plugin forum, if you fork the code so the new version does not require Python 2 compatibility, it does make life a lot easier for the person maintaining the code.

[JSWolf]

Originally Posted by DNSB

Much like the discussion in the Modify Epub plugin forum, if you fork the code so the new version does not require Python 2 compatibility, it does make life a lot easier for the person maintaining the code.

But the problem is that there is no way to show only Python 3 versions on 5+ and Python 2 versions on 4.23-. If that worked, then forking would be no problem.