05-25-2020, 11:42 AM | #1 |
Sigil Developer
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
|
Attention All Plugin Devs: Remove vestiges from python 2?
Hi all,
In an attempt to cleanup and simplify the code, I was planning to edit all the plugin launcher code and remove all vestiges of our former python 2.x support that still remain. Support for python 2 plugins in Sigil has long been removed so this should not impact any current plugins (or not many). Things like compatibility-utils, unipath, all the "from __future__" stuff, would all go away and lots of py2 only code as well in sigil_gumboc, sigil_bs4, etc. Given that Calibre now supports and desires python3 plugins as well, any plugins that shared code along these lines across the two projects should still work. I wanted to get feedback from plugin devs first, before starting this cleanup. And make sure they copy any needed code from plugin_launchers they still want into their own plugin projects before it starts going away. Please let me know if you have any concerns. Thanks, KevinH Last edited by KevinH; 05-27-2020 at 01:02 PM. |
05-27-2020, 01:02 PM | #2 |
Sigil Developer
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
|
Attention All Plugin Developers
Okay I took a shot at this. As far as I can tell no plugins outside my own use unipath.py and very very few use compatibility_utils.py.
That said our official testplugin does test for the presence of compatibility_utils. So it will have to be updated. KindleImport uses compatibility_utils.py but supplies its own via KindleUnpack core files. MarkDownImport uses compatibility_utils.py to just get iswindows but that is a one-line replacement fix. So far I have seen no serious issues against removing these vestiges of python 2.7 support. So if anyone has any issues or objections or reasons to keep it around, please let me know as soon as possible. Thanks! |
05-28-2020, 11:35 AM | #3 |
Sigil Developer
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
|
Okay, I grabbed a copy of all of our linked Plugins and ran checks on them to see what if anything they used from compatibility_utils.py and unipath.py
Here is what I found: Code:
Archive: ACE_0.2.2.zip .//ACE/plugin.py:from compatibility_utils import iswindows Archive: AddiBooksXML_v0.1.0.zip .//AddiBooksXML/plugin.py:from compatibility_utils import PY2, unicode_str .//AddiBooksXML/utilities.py:from compatibility_utils import PY2 .//AddiBooksXML/plugin.py:from unipath import pathof Archive: Borkify_0.2-plugin.zip .//Borkify/epub_utils.py:from compatibility_utils import text_type, binary_type, utf8_str, unicode_str, bord, bchr .//Borkify/epub_utils.py:import unipath .//Borkify/epub_utils.py:from unipath import pathof .//Borkify/epub_utils.py: files = unipath.walk(ebook_path) Archive: KindleGen_v0.5.5.zip .//KindleGen/mobi_split.py:from unipath import pathof Archive: KindleImport_v0.82.1.zip - Uses compatibility_utils and also internal version from kindleunpackcore - Uses internal version of unipath from kindleunpackcore .//KindleImport/gui_utilities.py:from compatibility_utils import PY2 .//KindleImport/gui_utilities.py:from compatibility_utils import unicode_str .//KindleImport/mobi_stuff.py:from compatibility_utils import PY2, bstr .//KindleImport/plugin.py:from compatibility_utils import PY2, unicode_str .//KindleImport/quickepub.py:from kindleunpackcore.compatibility_utils import unicode_str ... Archive: MarkdownImport_v0.1.2.zip .//Markdown/plugin.py:from compatibility_utils import iswindows Archive: ShadyCharacters_v0.1.1.zip from compatibility_utils import PY2 Archive: ShowSemantics_v0.1.5.zip from compatibility_utils import unquoteurl Archive: SubsetFonts_v0.2.3.zip .//SubsetFonts/plugin.py:from compatibility_utils import iswindows Archive: TagMechanic_v0.4.4.zip .//TagMechanic/plugin.py:from compatibility_utils import PY2 .//TagMechanic/dialogs.py:from compatibility_utils import PY2 Archive: icarus_v0.0.3.zip -Uses both compatibility_utils and unipath but supplies their own Archive: ACE_0.2.2.zip Archive: MarkdownImport_v0.1.2.zip Archive: SubsetFonts_v0.2.3.zip by replacing: from compatibility_utils import iswindows with: iswindows = sys.platform.startswith('win') Similarly ... We can update the following plugins easily: Archive: ShadyCharacters_v0.1.1.zip Archive: TagMechanic_v0.4.4.zip by replacing: from compatibility_utils import PY2 with: PY2 = sys.version_info[0] == 2 And that would just leave the following plugins which we could update by just adding a copy of compatibility_utils.py and/or unipath.py to them, (although KindleImport could just be updated by importing both from kindleunpackcore since it already includes both. Archive: AddiBooksXML_v0.1.0.zip Archive: Borkify_0.2-plugin.zip Archive: KindleGen_v0.5.5.zip Archive: KindleImport_v0.82.1.zip Archive: ShowSemantics_v0.1.5.zip Alternatively, we could keep both unipath.py and compatibility_utils.py and strip them down to just the used functions: iswindows, PY2, unquoteurl, unicode_str, pathof, etc, but that would leave some python2.7 vestiges still around in Sigil. Thoughts? KevinH Last edited by KevinH; 05-28-2020 at 11:56 AM. |
05-28-2020, 04:47 PM | #4 |
Grand Sorcerer
Posts: 27,465
Karma: 192992430
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
|
I think I'd prefer an approach that allowed the current versions of all plugins to continue to work with the current built-in framework compatibility features (stripped down to only the necessary functions if necessary). This would give the individual devs time to make the necessary updates to future-proof their plugins at their own pace.
Sort of a layered approach where hopefully plugins get a few new versions under their belts with their dependencies on compatibility_utils.py removed before the compatibility utils actually go away. Mainly because users don't always update plugins as regularly as they do Sigil. And if the utils disappear while they're still using older versions of plugins, we'll be dealing with a lot of "update the plugin" support threads/posts. |
05-28-2020, 05:06 PM | #5 |
Sigil Developer
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
|
Understood and agreed. Do you know of a way to officially make any import of compatibility_utils and unipath give a Deprecation warning of some sort so new plugins do not use them during the interim?
|
05-29-2020, 09:04 AM | #6 |
Grand Sorcerer
Posts: 27,465
Karma: 192992430
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
|
I might be able to rig up an import hook to add a deprecation warning, but using sys.path_hooks and sys.meta_path to intercept/substitute/augment/deny module/package imports got overly complicated in Python 3 in my opinion (not to mention that there's several function deprecations between python 3.4 and 3.6 ). It might be easier (and certainly less complicated) to "patch" the builtins __import__ function in the plugin framework to add a warning whenever importing anything from compatibility_utils. I can look into it if you like.
Last edited by DiapDealer; 05-29-2020 at 09:20 AM. |
05-29-2020, 09:44 AM | #7 |
Sigil Developer
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
|
Hmm, perhaps stripping it down but leaving it in place would be an easier option over deprecation then.
I just do not want us to be in the same situation in a few years and still stuck keeping those pieces around. I will take a shot at stripping it down and rewriting it to only use python3 when implementing things and see how that goes. If that doesn't work we can try some type of deprecation warning. Thanks |
05-29-2020, 11:15 AM | #8 |
Grand Sorcerer
Posts: 27,465
Karma: 192992430
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
|
I may be guilty of over-thinking this a bit. We could always add printed warnings directly to the various functions in compatibility_utils.py. Especially if it gets stripped down to the few remaining functions still being used by plugins.
|
05-29-2020, 11:55 AM | #9 |
Sigil Developer
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
|
Okay, I pushed all of my changes to master so we can use them to test against all plugins. We can always revert if we run into problems.
I actually kept all the methods in compatibility_utils.py and unipath.py but then simplified each a bit so that they only work for python3. Any python3 code importing them should continue to work unchanged. So all plugins should continue to work as long as they are using python3. Any running under python2.7 (which we removed a good while back) will fail but newer versions of Sigil do not support it anyway. When you get a free moment, please give it a try with KindleImport to make sure it works as expected. It should. If not, let me know and I will give it a shot to fix it. If this does not do the trick, we can revert to the full old versions of compatibility_utils.py and unipath.py for any release and then add in some type of "deprecated" warning. |
05-29-2020, 01:53 PM | #10 | |
Grand Sorcerer
Posts: 5,582
Karma: 22735033
Join Date: Dec 2010
Device: Kindle PW2
|
Quote:
Code:
from compatibility_utils import iswindows from compatibility_utils import unquoteurl |
|
05-29-2020, 02:28 PM | #11 |
Sigil Developer
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
|
|
05-29-2020, 03:16 PM | #12 |
Grand Sorcerer
Posts: 27,465
Karma: 192992430
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
|
Everything seems to work OK with the current KindleImport. Now I'll have to update the plugin to remove the dependencies for the framework version of the compatibility_utils. Since we're not supporting python 2 anymore, the plugin code itself shouldn't really have any need for the compatibility stuff. The core kindleunpack module included with the plugin is pretty-much self-contained.
I'm in the middle of an overhaul of the TagMechanic plugin right now anyway. Moving from tkinter to pyqt. Dropping support for Python2 was something I'd already factored into that. Last edited by DiapDealer; 05-29-2020 at 03:18 PM. |
Thread Tools | Search this Thread |
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
Is the environment Python 2 or Python 3? | halloleo | Library Management | 11 | 04-07-2020 03:47 AM |
Python 2 to Python 3, Polyglot & Plugins | DaltonST | Calibre | 18 | 04-24-2019 12:10 AM |
Python help please | AndyW1691 | Library Management | 3 | 02-08-2012 09:51 AM |
Remove Books - Restoring Confirm to remove books | Caffey | Calibre | 6 | 09-20-2010 09:23 AM |
RFE: Remove remove tags in bulk edit | magphil | Calibre | 0 | 08-11-2009 10:37 AM |