Attn plugin devs: porting to python 3 - Page 3

jhowell · 12-18-2019, 03:57 AM

Quote:

Originally Posted by kovidgoyal

I uploaded 4.99.1 with fixes for all reported issues.

That fixed my import error. Thank you!

thiago.eec · 12-18-2019, 08:24 AM

Quote:

Originally Posted by eschwartz

Indeed, the majority of issues you're likely to have are related to encoding/decoding strings, and tools like modernize/futurize usually cannot help with that. You'll just need to resolve them via (usually) simple trial and error.

All good now! No problems with the multi-threading code. Just some other missing encoding/decoding issues.
Both plugins are working 100% on python 3 now. The open dialog and translation are fixed too.

Thanks for the help.

DaltonST · 12-18-2019, 11:31 AM

Kovid,

Given that Calibre 4.99 will silently fail at startup with no indication as to the cause if a UI plugin that is not compatible with Python 3 is initialized, and given that many users have many plugins, IMHO it might be prudent to add a new attribute to InterfaceActionBase that defaults to False and means that the respective UI plugin is "safe" to use with Python 3 Calibre only when the plugin returns True for that new attribute. This should help to avoid needless user mass confusion and panic in the future.

DaltonST

kovidgoyal · 12-18-2019, 11:49 AM

I dont see how calibre 4.99 will fail at startup regardless of what UI plugins do, unless those UI plugins mess with sys.path, or other global interpreter state, which is the case with all versions of calibre.

thiago.eec · 12-18-2019, 12:33 PM

Quote:

Originally Posted by DaltonST

...Calibre 4.99 will silently fail at startup with no indication as to the cause if a UI plugin that is not compatible with Python 3 is initialized

Calibre starts normally here. In the debug log I can see all the plugins that failed to load. Also, some plugins load, but fail when I click on then.

DaltonST · 12-18-2019, 12:36 PM

Quote:

Originally Posted by kovidgoyal

I dont see how calibre 4.99 will fail at startup regardless of what UI plugins do, unless those UI plugins mess with sys.path, or other global interpreter state, which is the case with all versions of calibre.

See your answer to this post:https://www.mobileread.com/forums/sh...82&postcount=9

which you provided here: https://www.mobileread.com/forums/showpost.php?p=3929798&postcount=15

Quote:

@DaltonST: Likely one of those plugins is causing the issue them, run calibre as

calibre --ignore-plugins

You were correct. calibre --ignore-plugins allowed 4.99 to start up. Otherwise, back to my original post at https://www.mobileread.com/forums/sh...82&postcount=9

Remember that virtually ALL plugins will fail in 4.99 unless they are ported to Python 3 first. Users will have no idea why Calibre silently dies on them. Hence, user mass confusion.

DaltonST

thiago.eec · 12-18-2019, 12:50 PM

Hi @kovidgoyal

Just to be sure: is the metadata download fully functional on 4.99.1? I was trying to port a plugin for it, but I'm not sure it is working, since even the bundled plugins are failing.
The log points out to some problem with Qt geometry:

Spoiler:

Also, I just noticed an error with QWebView that is showing both on calibre 4.6 and 4.99.1:

Spoiler:

DaltonST · 12-18-2019, 12:54 PM

Quote:

Originally Posted by thiago.eec

Calibre starts normally here. In the debug log I can see all the plugins that failed to load. Also, some plugins load, but fail when I click on then.

See my original debug log. Calibre silently died AFTER initialization of all 20 of my installed plugins, with nothing in the debug log. Hence, my first post in this thread.

DaltonST

JimmXinu · 12-18-2019, 01:00 PM

(Tangentially off topic?)

So while working on this, I'm wondering: How common are pre-3.48 Calibre users now?

I'm thinking I might make the min Calibre version for all my plugins 3.48 and start removing some old cruft and the PyQt4 code...

DiapDealer · 12-18-2019, 01:18 PM

Looks like I have a little bit of playing to do.

eschwartz · 12-18-2019, 02:25 PM

Quote:

Originally Posted by DaltonST

See my original debug log. Calibre silently died AFTER initialization of all 20 of my installed plugins, with nothing in the debug log. Hence, my first post in this thread.

I've been running python3 in various stages for the better part of a year with lots of plugins installed, and no plugin-related issues.

Whatever *your* plugin-related issue is, it somehow caused a function in Calibre itself to go missing so that Calibre itself could not find it and failed at startup.

I'm baffled what your plugins could do that caused such an issue. It's certainly not something that most plugins should be concerned about.

If you can test your profile with a single plugin enabled at a time, and figure out what actually caused the crash to happen (or even a way for someone else to repeat it) that would be great and we could investigate the cause, and determine whether it's a sufficiently painful issue to require skipping plugins.

But at the moment, we are working on vague theories and insinuations, so I think we should go with Kovid's gut instinct and not restrict anything here.

eschwartz · 12-18-2019, 02:31 PM

Quote:

Originally Posted by JimmXinu

(Tangentially off topic?)

So while working on this, I'm wondering: How common are pre-3.48 Calibre users now?

I'm thinking I might make the min Calibre version for all my plugins 3.48 and start removing some old cruft and the PyQt4 code...

The pyqt4 code should only be needed on 1.48, which basically means Windows XP users I think.

I'm not sure there's any advantage in requiring anything stricter than >=2.0.0

It's possible there are people still using 2.x because they like the old content server or don't like the fact that running calibre-server in the background prevents starting the GUI.

JimmXinu · 12-18-2019, 02:40 PM

Another minor issue: When running debug from Preferences -> Restart in Debug, after restart, when presented with the 'Running in debug, log saved to this temp file', if I clicked the 'Copy to Clipboard' button, then okay, there's an error in the log apparently from the 'Copy to Clipboard'. But the text of the dialog is copied to clipboard.

See log:

Spoiler:

DiapDealer · 12-18-2019, 02:49 PM

What version did UI plugins first get introduced to calibre?

DiapDealer · 12-18-2019, 02:53 PM

Quote:

Originally Posted by DiapDealer

What version did UI plugins first get introduced to calibre?

Never mind. Found it my own big self.

>= 0.8.60

12-18-2019, 11:31 AM	#33
DaltonST Deviser Posts: 2,265 Karma: 2090983 Join Date: Aug 2013 Location: Texas Device: none	Kovid, Given that Calibre 4.99 will silently fail at startup with no indication as to the cause if a UI plugin that is not compatible with Python 3 is initialized, and given that many users have many plugins, IMHO it might be prudent to add a new attribute to InterfaceActionBase that defaults to False and means that the respective UI plugin is "safe" to use with Python 3 Calibre only when the plugin returns True for that new attribute. This should help to avoid needless user mass confusion and panic in the future. DaltonST Last edited by DaltonST; 12-18-2019 at 11:37 AM.

12-18-2019, 12:50 PM	#37
thiago.eec Wizard Posts: 1,216 Karma: 1419583 Join Date: Dec 2016 Location: Goiânia - Brazil Device: iPad, Kindle Paperwhite, Kindle Oasis	Hi @kovidgoyal Just to be sure: is the metadata download fully functional on 4.99.1? I was trying to port a plugin for it, but I'm not sure it is working, since even the bundled plugins are failing. The log points out to some problem with Qt geometry: Spoiler: calibre 4.99.1 embedded-python: True is64bit: False Windows-10-10.0.18362-SP0 Windows ('32bit', 'WindowsPE') 32bit process running on 64bit windows ('Windows', '10', '10.0.18362') Python 3.8.0 Windows: ('10', '10.0.18362', 'SP0', '') Interface language: pt_BR Failed to initialize plugin: 'C:\\Users\\go_th\\AppData\\Roaming\\calibre\\plug ins\\DeDRM.zip' Traceback (most recent call last): File "ui.py", line 703, in initialize_plugins File "ui.py", line 61, in load_plugin File "zipplugin.py", line 202, in load File "__init__.py", line 127, in import_module File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 655, in _load_unlocked File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible File "zipplugin.py", line 179, in load_module File "calibre_plugins.dedrm.__init__", line 164 print u"{0} v{1}: Copying needed library files from plugin's zip".format(PLUGIN_NAME, PLUGIN_VERSION) ^ SyntaxError: invalid syntax Failed to initialize plugin: 'C:\\Users\\go_th\\AppData\\Roaming\\calibre\\plug ins\\Goodreads.zip' Traceback (most recent call last): File "ui.py", line 703, in initialize_plugins File "ui.py", line 61, in load_plugin File "zipplugin.py", line 202, in load File "__init__.py", line 127, in import_module File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 655, in _load_unlocked File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible File "zipplugin.py", line 184, in load_module File "calibre_plugins.goodreads.__init__", line 11, in <module> ImportError: cannot import name 'quote' from 'urllib' (C:\Users\go_th\Documents\Calibre Portable\Calibre\\app\pylib.zip\urllib\__init__.py c) Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (1, 25, 0) && Package KFX (from KFX Input) (1, 25, 0) && ACE (1, 1, 0) && Count Pages (1, 9, 0) && Diaps Editing Toolbag (0, 3, 5) && EpubCheck (0, 2, 1) && EpubMerge (2, 7, 0) && EpubSplit (2, 8, 0) && Find Duplicates (1, 6, 3) && Goodreads Sync (1, 12, 1) && KFX metadata reader (from KFX Input) (1, 25, 0) && KFX Input (1, 25, 0) && Set KFX metadata (from KFX Output) (1, 35, 0) && KFX Output (1, 35, 0) && KindleUnpack - The Plugin (0, 81, 5) && LanguageTool (0, 1, 0) && Open With (1, 5, 10) && ScrambleEbook (0, 4, 1) && Skoob Books (1, 3, 1) && Skoob Sync (0, 3, 0) && TTS to MP3 (0, 1, 3) Turning on automatic hidpi scaling devicePixelRatio: 1.0 logicalDpi: 120.0 x 120.0 physicalDpi: 141.7674418604651 x 141.4020618556701 Using calibre Qt style: True [0.00] Starting up... [0.02] Showing splash screen... [0.55] splash screen shown [0.55] Initializing db... [0.61] db initialized [0.61] Constructing main UI... Traceback (most recent call last): File "ui.py", line 157, in __init__ File "ui.py", line 171, in init_iaction File "__init__.py", line 612, in load_actual_plugin File "__init__.py", line 127, in import_module File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 655, in _load_unlocked File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible File "zipplugin.py", line 184, in load_module File "calibre_plugins.epubmerge.epubmerge_plugin", line 14, in <module> ModuleNotFoundError: No module named 'StringIO' Traceback (most recent call last): File "ui.py", line 157, in __init__ File "ui.py", line 171, in init_iaction File "__init__.py", line 612, in load_actual_plugin File "__init__.py", line 127, in import_module File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 655, in _load_unlocked File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible File "zipplugin.py", line 184, in load_module File "calibre_plugins.epubsplit.epubsplit_plugin", line 14, in <module> ModuleNotFoundError: No module named 'ConfigParser' Traceback (most recent call last): File "ui.py", line 157, in __init__ File "ui.py", line 171, in init_iaction File "__init__.py", line 612, in load_actual_plugin File "__init__.py", line 127, in import_module File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 655, in _load_unlocked File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible File "zipplugin.py", line 184, in load_module File "calibre_plugins.goodreads_sync.action", line 25, in <module> File "zipplugin.py", line 184, in load_module File "calibre_plugins.goodreads_sync.core", line 5, in <module> ImportError: cannot import name 'tostring' from '_elementtree' (C:\Users\go_th\Documents\Calibre Portable\Calibre\app\bin\_elementtree.pyd) Traceback (most recent call last): File "ui.py", line 157, in __init__ File "ui.py", line 171, in init_iaction File "__init__.py", line 612, in load_actual_plugin File "__init__.py", line 127, in import_module File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 655, in _load_unlocked File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible File "zipplugin.py", line 179, in load_module File "calibre_plugins.kindleunpack_plugin.action", line 268 except Exception, e: ^ SyntaxError: invalid syntax PyQt5.QtWebKitWidgets.QWebView failed PyQt5.QtWebEngineWidgets.QWebEngineView failed Traceback (most recent call last): File "ui.py", line 157, in __init__ File "ui.py", line 171, in init_iaction File "__init__.py", line 612, in load_actual_plugin File "__init__.py", line 127, in import_module File "<frozen importlib._bootstrap>", line 1014, in _gcd_import File "<frozen importlib._bootstrap>", line 991, in _find_and_load File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 655, in _load_unlocked File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible File "zipplugin.py", line 179, in load_module File "calibre_plugins.tts_to_mp3_plugin.uiaction", line 129 raise SelectedBookError, errmsg ^ SyntaxError: invalid syntax Traceback (most recent call last): File "ui.py", line 412, in initialize File "calibre_plugins.find_duplicates.action", line 54, in initialization_complete File "calibre_plugins.find_duplicates.duplicates", line 69, in __init__ File "calibre_plugins.find_duplicates.config", line 111, in get_exemption_lists NameError: name 'xrange' is not defined [2.03] main UI initialized... [2.03] Hiding splash screen [2.53] splash screen hidden [2.53] Started up in 2.53 seconds with 981 books Metadata sources cache was recently updated not updating again QWindowsWindow::setGeometry: Unable to set geometry 604x218+676+354 on QWidgetWindow/'DialogWindow'. Resulting geometry: 604x246+676+354 (frame: 9, 38, 9, 9, custom margin: 0, 0, 0, 0, minimum size: 530x218, maximum size: 16777215x16777215). Also, I just noticed an error with QWebView that is showing both on calibre 4.6 and 4.99.1: Spoiler: PyQt5.QtWebKitWidgets.QWebView failed PyQt5.QtWebEngineWidgets.QWebEngineView failed

12-18-2019, 02:49 PM	#44
DiapDealer Grand Sorcerer Posts: 28,575 Karma: 204127028 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	What version did UI plugins first get introduced to calibre? Last edited by DiapDealer; 12-18-2019 at 02:51 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Plugin Devs: Should we include a License or COPYING file?	KevinH	Plugins	3	08-15-2017 01:09 PM
Attn plugin developers: calibre moving to Qt 5	kovidgoyal	Plugins	157	01-01-2015 11:22 PM
Can I develop a plugin in a launguage other than Python?	jamawg	Development	2	06-13-2014 12:38 AM
Calibre in a Python 3.2 world, attn: Kovid	Kevin McAleavey	Calibre	8	01-09-2012 05:49 PM
How do I Create a Python Plugin?	Sydney's Mom	Plugins	25	01-27-2010 06:26 AM

12-18-2019, 11:49 AM	#34
kovidgoyal creator of calibre Posts: 45,364 Karma: 27230406 Join Date: Oct 2006 Location: Mumbai, India Device: Various	I dont see how calibre 4.99 will fail at startup regardless of what UI plugins do, unless those UI plugins mess with sys.path, or other global interpreter state, which is the case with all versions of calibre.

12-18-2019, 01:00 PM	#39
JimmXinu Plugin Developer Posts: 6,973 Karma: 4604635 Join Date: Dec 2011 Location: Midwest USA Device: Kobo Clara Colour running KOReader	(Tangentially off topic?) So while working on this, I'm wondering: How common are pre-3.48 Calibre users now? I'm thinking I might make the min Calibre version for all my plugins 3.48 and start removing some old cruft and the PyQt4 code...

12-18-2019, 01:18 PM	#40
DiapDealer Grand Sorcerer Posts: 28,575 Karma: 204127028 Join Date: Jan 2010 Device: Nexus 7, Kindle Fire HD	Looks like I have a little bit of playing to do.