How is this possible?

karl1c · 03-25-2019, 07:13 PM

I wrote a script to sync metadata from Calibre to KyBook3, which works well. I decided it should really be a plugin and downloaded the interface plugin demo and got it working as a plugin. Went to open Calibre the next day and the plugin refused to load (although calibre-debug -g says it has).

Thinking I must have changed something, I commented out code until I got it working again and started putting the code back in bit by bit. But it kept loading and working until all the code was back in. (The md5s of the file were the same.)

Does anyone know what could cause this?

theducks · 03-25-2019, 08:28 PM

Ya need to post the debug log (please wrap in spoiler),
And maybe attach your zipped script so the gurus can see what you did

karl1c · 03-26-2019, 07:02 AM

The plugin was working last night on my MacBook. I went to produce the log file this morning and did this:

> calibre-customize -b .
> Plugin updated: KyBook3 Sync (1, 0, 1)

Started Calibre and Restart in Debug Mode from the toolbar and got this:

Spoiler:

So it seems it is intermittently failing to find urllib3. (Or, possibly, other modules when it fails.) As I said, if I comment out the code and put it back in again the problem goes away and randomly comes back again. Is it a memory issue? As I'm loading quite a lot of Python code to get the Requests module.

karl1c · 03-26-2019, 07:08 AM

BTW, urllib3 is being loaded by the Requests module rather than 'directly' from the plugin scripts.

kovidgoyal · 03-26-2019, 01:31 PM

You cannot depend on random python libraries unless yo bundle them with your plugin. requests is not part of calibre's python environment. Use mechanize or the stdlib urllib modules instead.

karl1c · 03-26-2019, 03:16 PM

@kovidgoyal It is bundled in the plugin. That's why it does work intermittently. :-(

karl1c · 03-26-2019, 07:00 PM

It seems the problem was that Calibre would randomly refuse to load the python modules included in the plugin zip file. I resolved the problem by removing the need for the modules.

davidfor · 03-26-2019, 10:21 PM

I did have a look at this last night, but ran out of time.

The problem I saw was that the import were not finding the modules. The times I have done something like this, I needed to prefix the included module name with the plugins details. An example I have is:

Code:

import calibre_plugins.goodreads_sync.httplib2 as httplib2

This is inside another module in the package.

I made this change for a few of the statements, but, didn't have time to do all.

Unfortunately, that would mean going through all those modules and changing the import statements. There might be a better way with a load library statement or using relative paths, but, I haven't experimented.

03-25-2019, 07:13 PM	#1
karl1c Connoisseur Posts: 56 Karma: 15570 Join Date: Jul 2017 Device: iPad	How is this possible? I wrote a script to sync metadata from Calibre to KyBook3, which works well. I decided it should really be a plugin and downloaded the interface plugin demo and got it working as a plugin. Went to open Calibre the next day and the plugin refused to load (although calibre-debug -g says it has). Thinking I must have changed something, I commented out code until I got it working again and started putting the code back in bit by bit. But it kept loading and working until all the code was back in. (The md5s of the file were the same.) Does anyone know what could cause this?

03-26-2019, 07:02 AM	#3
karl1c Connoisseur Posts: 56 Karma: 15570 Join Date: Jul 2017 Device: iPad	Log file The plugin was working last night on my MacBook. I went to produce the log file this morning and did this: > calibre-customize -b . > Plugin updated: KyBook3 Sync (1, 0, 1) Started Calibre and Restart in Debug Mode from the toolbar and got this: Spoiler: Traceback (most recent call last): File "site-packages/calibre/gui2/ui.py", line 157, in __init__ File "site-packages/calibre/gui2/ui.py", line 171, in init_iaction File "site-packages/calibre/customize/__init__.py", line 612, in load_actual_plugin File "lib/python2.7/importlib/__init__.py", line 37, in import_module File "site-packages/calibre/customize/zipplugin.py", line 184, in load_module File "calibre_plugins.kybook3_sync.ui", line 18, in <module> File "site-packages/calibre/customize/zipplugin.py", line 184, in load_module File "calibre_plugins.kybook3_sync.main", line 23, in <module> File "site-packages/calibre/customize/zipplugin.py", line 184, in load_module File "calibre_plugins.kybook3_sync.cal2ky3", line 26, in <module> File "site-packages/calibre/customize/zipplugin.py", line 184, in load_module File "calibre_plugins.kybook3_sync.requests.__init_ _", line 43, in <module> ImportError: No module named urllib3 calibre Debug log calibre 3.34 embedded-python: True is64bit: True Darwin-17.3.0-x86_64-i386-64bit Darwin ('64bit', '') ('Darwin', '17.3.0', 'Darwin Kernel Version 17.3.0: Thu Nov 9 18:09:22 PST 2017; root:xnu-4570.31.3~1/RELEASE_X86_64') Python 2.7.12 OSX: ('10.13.2', ('', '', ''), 'x86_64') Interface language: en_GB Successfully initialized third party plugins: KyBook3 Sync (1, 0, 1) calibre 3.34 embedded-python: True is64bit: True Darwin-17.3.0-x86_64-i386-64bit Darwin ('64bit', '') ('Darwin', '17.3.0', 'Darwin Kernel Version 17.3.0: Thu Nov 9 18:09:22 PST 2017; root:xnu-4570.31.3~1/RELEASE_X86_64') Python 2.7.12 OSX: ('10.13.2', ('', '', ''), 'x86_64') Interface language: en_GB Successfully initialized third party plugins: KyBook3 Sync (1, 0, 1) devicePixelRatio: 1.0 logicalDpi: 72.0 x 72.0 physicalDpi: 114.000001712 x 114.000001712 Using calibre Qt style: True [0.00] Starting up... [0.02] Showing splash screen... [0.22] splash screen shown [0.22] Initializing db... [0.24] db initialized [0.24] Constructing main UI... [2.04] main UI initialized... [2.04] Hiding splash screen [2.08] splash screen hidden [2.08] Started up in 2.08 seconds with 8 books So it seems it is intermittently failing to find urllib3. (Or, possibly, other modules when it fails.) As I said, if I comment out the code and put it back in again the problem goes away and randomly comes back again. Is it a memory issue? As I'm loading quite a lot of Python code to get the Requests module. Last edited by karl1c; 03-26-2019 at 06:57 PM.

03-26-2019, 10:21 PM	#8
davidfor Grand Sorcerer Posts: 24,905 Karma: 47303824 Join Date: Jul 2011 Location: Sydney, Australia Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos	I did have a look at this last night, but ran out of time. The problem I saw was that the import were not finding the modules. The times I have done something like this, I needed to prefix the included module name with the plugins details. An example I have is: Code: import calibre_plugins.goodreads_sync.httplib2 as httplib2 This is inside another module in the package. I made this change for a few of the statements, but, didn't have time to do all. Unfortunately, that would mean going through all those modules and changing the import statements. There might be a better way with a load library statement or using relative paths, but, I haven't experimented.

03-25-2019, 08:28 PM	#2
theducks Well trained by Cats Posts: 31,047 Karma: 60358908 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	Ya need to post the debug log (please wrap in spoiler), And maybe attach your zipped script so the gurus can see what you did

03-26-2019, 07:08 AM	#4
karl1c Connoisseur Posts: 56 Karma: 15570 Join Date: Jul 2017 Device: iPad	BTW, urllib3 is being loaded by the Requests module rather than 'directly' from the plugin scripts.

03-26-2019, 01:31 PM	#5
kovidgoyal creator of calibre Posts: 45,345 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	You cannot depend on random python libraries unless yo bundle them with your plugin. requests is not part of calibre's python environment. Use mechanize or the stdlib urllib modules instead.

03-26-2019, 03:16 PM	#6
karl1c Connoisseur Posts: 56 Karma: 15570 Join Date: Jul 2017 Device: iPad	@kovidgoyal It is bundled in the plugin. That's why it does work intermittently. :-(

03-26-2019, 07:00 PM	#7
karl1c Connoisseur Posts: 56 Karma: 15570 Join Date: Jul 2017 Device: iPad	It seems the problem was that Calibre would randomly refuse to load the python modules included in the plugin zip file. I resolved the problem by removing the need for the modules.

Advert

Advert