Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Sigil > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 05-25-2020, 11:42 AM   #1
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
Attention All Plugin Devs: Remove vestiges from python 2?

Hi all,
In an attempt to cleanup and simplify the code, I was planning to edit all the plugin launcher code and remove all vestiges of our former python 2.x support that still remain. Support for python 2 plugins in Sigil has long been removed so this should not impact any current plugins (or not many).

Things like compatibility-utils, unipath, all the "from __future__" stuff, would all go away and lots of py2 only code as well in sigil_gumboc, sigil_bs4, etc.

Given that Calibre now supports and desires python3 plugins as well, any plugins that shared code along these lines across the two projects should still work.

I wanted to get feedback from plugin devs first, before starting this cleanup.
And make sure they copy any needed code from plugin_launchers they still want into their own plugin projects before it starts going away.

Please let me know if you have any concerns.

Thanks,

KevinH

Last edited by KevinH; 05-27-2020 at 01:02 PM.
KevinH is offline   Reply With Quote
Old 05-27-2020, 01:02 PM   #2
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
Attention All Plugin Developers

Okay I took a shot at this. As far as I can tell no plugins outside my own use unipath.py and very very few use compatibility_utils.py.

That said our official testplugin does test for the presence of compatibility_utils. So it will have to be updated.

KindleImport uses compatibility_utils.py but supplies its own via KindleUnpack core files.

MarkDownImport uses compatibility_utils.py to just get iswindows but that is a one-line replacement fix.

So far I have seen no serious issues against removing these vestiges of python 2.7 support.

So if anyone has any issues or objections or reasons to keep it around, please let me know as soon as possible.

Thanks!
KevinH is offline   Reply With Quote
Old 05-28-2020, 11:35 AM   #3
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
Okay, I grabbed a copy of all of our linked Plugins and ran checks on them to see what if anything they used from compatibility_utils.py and unipath.py

Here is what I found:

Code:
Archive:  ACE_0.2.2.zip
.//ACE/plugin.py:from compatibility_utils import iswindows


Archive:  AddiBooksXML_v0.1.0.zip
.//AddiBooksXML/plugin.py:from compatibility_utils import PY2, unicode_str
.//AddiBooksXML/utilities.py:from compatibility_utils import PY2
.//AddiBooksXML/plugin.py:from unipath import pathof


Archive:  Borkify_0.2-plugin.zip
.//Borkify/epub_utils.py:from compatibility_utils import text_type, binary_type, utf8_str, unicode_str, bord, bchr
.//Borkify/epub_utils.py:import unipath
.//Borkify/epub_utils.py:from unipath import pathof
.//Borkify/epub_utils.py:    files = unipath.walk(ebook_path)


Archive:  KindleGen_v0.5.5.zip
.//KindleGen/mobi_split.py:from unipath import pathof


Archive:  KindleImport_v0.82.1.zip
  - Uses compatibility_utils and also internal version from kindleunpackcore
  - Uses internal version of unipath from kindleunpackcore
.//KindleImport/gui_utilities.py:from compatibility_utils import PY2
.//KindleImport/gui_utilities.py:from compatibility_utils import unicode_str
.//KindleImport/mobi_stuff.py:from compatibility_utils import PY2, bstr
.//KindleImport/plugin.py:from compatibility_utils import PY2, unicode_str
.//KindleImport/quickepub.py:from kindleunpackcore.compatibility_utils import unicode_str
...


Archive:  MarkdownImport_v0.1.2.zip
.//Markdown/plugin.py:from compatibility_utils import iswindows


Archive:  ShadyCharacters_v0.1.1.zip
from compatibility_utils import PY2


Archive:  ShowSemantics_v0.1.5.zip
from compatibility_utils import unquoteurl


Archive:  SubsetFonts_v0.2.3.zip
.//SubsetFonts/plugin.py:from compatibility_utils import iswindows


Archive:  TagMechanic_v0.4.4.zip
.//TagMechanic/plugin.py:from compatibility_utils import PY2
.//TagMechanic/dialogs.py:from compatibility_utils import PY2


Archive:  icarus_v0.0.3.zip
  -Uses both compatibility_utils and unipath but supplies their own
So it appears we can update the following plugins easily:

Archive: ACE_0.2.2.zip
Archive: MarkdownImport_v0.1.2.zip
Archive: SubsetFonts_v0.2.3.zip

by replacing:

from compatibility_utils import iswindows

with:

iswindows = sys.platform.startswith('win')

Similarly ...

We can update the following plugins easily:

Archive: ShadyCharacters_v0.1.1.zip
Archive: TagMechanic_v0.4.4.zip

by replacing:

from compatibility_utils import PY2

with:

PY2 = sys.version_info[0] == 2

And that would just leave the following plugins which we could update by just adding a copy of compatibility_utils.py and/or unipath.py to them, (although KindleImport could just be updated by importing both from kindleunpackcore since it already includes both.

Archive: AddiBooksXML_v0.1.0.zip
Archive: Borkify_0.2-plugin.zip
Archive: KindleGen_v0.5.5.zip
Archive: KindleImport_v0.82.1.zip
Archive: ShowSemantics_v0.1.5.zip

Alternatively, we could keep both unipath.py and compatibility_utils.py and strip them down to just the used functions: iswindows, PY2, unquoteurl, unicode_str, pathof, etc, but that would leave some python2.7 vestiges still around in Sigil.

Thoughts?

KevinH

Last edited by KevinH; 05-28-2020 at 11:56 AM.
KevinH is offline   Reply With Quote
Old 05-28-2020, 04:47 PM   #4
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 27,465
Karma: 192992430
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
I think I'd prefer an approach that allowed the current versions of all plugins to continue to work with the current built-in framework compatibility features (stripped down to only the necessary functions if necessary). This would give the individual devs time to make the necessary updates to future-proof their plugins at their own pace.

Sort of a layered approach where hopefully plugins get a few new versions under their belts with their dependencies on compatibility_utils.py removed before the compatibility utils actually go away.

Mainly because users don't always update plugins as regularly as they do Sigil. And if the utils disappear while they're still using older versions of plugins, we'll be dealing with a lot of "update the plugin" support threads/posts.
DiapDealer is offline   Reply With Quote
Old 05-28-2020, 05:06 PM   #5
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
Understood and agreed. Do you know of a way to officially make any import of compatibility_utils and unipath give a Deprecation warning of some sort so new plugins do not use them during the interim?
KevinH is offline   Reply With Quote
Old 05-29-2020, 09:04 AM   #6
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 27,465
Karma: 192992430
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Quote:
Originally Posted by KevinH View Post
Understood and agreed. Do you know of a way to officially make any import of compatibility_utils and unipath give a Deprecation warning of some sort so new plugins do not use them during the interim?
I might be able to rig up an import hook to add a deprecation warning, but using sys.path_hooks and sys.meta_path to intercept/substitute/augment/deny module/package imports got overly complicated in Python 3 in my opinion (not to mention that there's several function deprecations between python 3.4 and 3.6 ). It might be easier (and certainly less complicated) to "patch" the builtins __import__ function in the plugin framework to add a warning whenever importing anything from compatibility_utils. I can look into it if you like.

Last edited by DiapDealer; 05-29-2020 at 09:20 AM.
DiapDealer is offline   Reply With Quote
Old 05-29-2020, 09:44 AM   #7
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
Hmm, perhaps stripping it down but leaving it in place would be an easier option over deprecation then.

I just do not want us to be in the same situation in a few years and still stuck keeping those pieces around.

I will take a shot at stripping it down and rewriting it to only use python3 when implementing things and see how that goes. If that doesn't work we can try some type of deprecation warning.

Thanks
KevinH is offline   Reply With Quote
Old 05-29-2020, 11:15 AM   #8
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 27,465
Karma: 192992430
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
I may be guilty of over-thinking this a bit. We could always add printed warnings directly to the various functions in compatibility_utils.py. Especially if it gets stripped down to the few remaining functions still being used by plugins.
DiapDealer is offline   Reply With Quote
Old 05-29-2020, 11:55 AM   #9
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
Okay, I pushed all of my changes to master so we can use them to test against all plugins. We can always revert if we run into problems.

I actually kept all the methods in compatibility_utils.py and unipath.py but then simplified each a bit so that they only work for python3.

Any python3 code importing them should continue to work unchanged. So all plugins should continue to work as long as they are using python3.

Any running under python2.7 (which we removed a good while back) will fail but newer versions of Sigil do not support it anyway.

When you get a free moment, please give it a try with KindleImport to make sure it works as expected. It should. If not, let me know and I will give it a shot to fix it.

If this does not do the trick, we can revert to the full old versions of compatibility_utils.py and unipath.py for any release and then add in some type of "deprecated" warning.
KevinH is offline   Reply With Quote
Old 05-29-2020, 01:53 PM   #10
Doitsu
Grand Sorcerer
Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.Doitsu ought to be getting tired of karma fortunes by now.
 
Doitsu's Avatar
 
Posts: 5,582
Karma: 22735033
Join Date: Dec 2010
Device: Kindle PW2
Quote:
Originally Posted by KevinH View Post
I actually kept all the methods in compatibility_utils.py and unipath.py but then simplified each a bit so that they only work for python3.
AFAIK, I only used:

Code:
from compatibility_utils import iswindows
from compatibility_utils import unquoteurl
in some of my plugins and they all still work fine with latest build.
Doitsu is offline   Reply With Quote
Old 05-29-2020, 02:28 PM   #11
KevinH
Sigil Developer
KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.KevinH ought to be getting tired of karma fortunes by now.
 
Posts: 7,506
Karma: 5433350
Join Date: Nov 2009
Device: many
Thanks! It is nice to know I have not broken anything!

Quote:
Originally Posted by Doitsu View Post
AFAIK, I only used:

Code:
from compatibility_utils import iswindows
from compatibility_utils import unquoteurl
in some of my plugins and they all still work fine with latest build.
KevinH is offline   Reply With Quote
Old 05-29-2020, 03:16 PM   #12
DiapDealer
Grand Sorcerer
DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.DiapDealer ought to be getting tired of karma fortunes by now.
 
DiapDealer's Avatar
 
Posts: 27,465
Karma: 192992430
Join Date: Jan 2010
Device: Nexus 7, Kindle Fire HD
Everything seems to work OK with the current KindleImport. Now I'll have to update the plugin to remove the dependencies for the framework version of the compatibility_utils. Since we're not supporting python 2 anymore, the plugin code itself shouldn't really have any need for the compatibility stuff. The core kindleunpack module included with the plugin is pretty-much self-contained.

I'm in the middle of an overhaul of the TagMechanic plugin right now anyway. Moving from tkinter to pyqt. Dropping support for Python2 was something I'd already factored into that.

Last edited by DiapDealer; 05-29-2020 at 03:18 PM.
DiapDealer is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Is the environment Python 2 or Python 3? halloleo Library Management 11 04-07-2020 03:47 AM
Python 2 to Python 3, Polyglot & Plugins DaltonST Calibre 18 04-24-2019 12:10 AM
Python help please AndyW1691 Library Management 3 02-08-2012 09:51 AM
Remove Books - Restoring Confirm to remove books Caffey Calibre 6 09-20-2010 09:23 AM
RFE: Remove remove tags in bulk edit magphil Calibre 0 08-11-2009 10:37 AM


All times are GMT -4. The time now is 04:45 AM.


MobileRead.com is a privately owned, operated and funded community.