Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 12-17-2019, 03:18 PM   #16
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 3,917
Karma: 637184
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
I'm getting an error when trying to open a file dialog. It happens with both Add Book and Create Library. Both with beta and running from source. Pretty sure it's unrelated to plugins.

Debug log attached.

(Sorry if there's a better place to report this sort of thing.)
Attached Files
File Type: txt tmprfugkjkz.txt (2.7 KB, 19 views)
JimmXinu is offline   Reply With Quote
Old 12-17-2019, 04:08 PM   #17
jhowell
Wizard
jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.
 
jhowell's Avatar
 
Posts: 3,369
Karma: 32493247
Join Date: Nov 2011
Location: Tampa Bay, Florida
Device: Oasis 2, Fire, iPad Air 2, Nexus 7
I have a plugin that uses the following import:

Code:
from Crypto.Cipher import AES
This loads fine under calibre 4.6.0 portable, but under 4.99.0 portable it produces this error:

Spoiler:
Code:
Traceback (most recent call last):
  File "ui.py", line 157, in __init__
  File "ui.py", line 171, in init_iaction
  File "__init__.py", line 612, in load_actual_plugin
  File "__init__.py", line 127, in import_module
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "zipplugin.py", line 184, in load_module
  File "calibre_plugins.overdrive_link.action", line 7, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "__init__.py", line 27, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "_mode_ecb.py", line 35, in <module>
  File "_raw_api.py", line 299, in load_pycryptodome_raw_lib
OSError: Cannot load native module 'Crypto.Cipher._raw_ecb': Trying '_raw_ecb.cp38-win32.pyd': Could not find module 'C:\calibre-beta\Calibre Portable\Calibre\app\pylib.zip\Crypto\Cipher\_raw_ecb.cp38-win32.pyd'. Try using the full path with constructor syntax., Trying '_raw_ecb.pyd': Could not find module 'C:\calibre-beta\Calibre Portable\Calibre\app\pylib.zip\Crypto\Cipher\_raw_ecb.pyd'. Try using the full path with constructor syntax.
jhowell is online now   Reply With Quote
Advert
Old 12-17-2019, 05:14 PM   #18
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 3,917
Karma: 637184
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
Another issue I've found. (Win10, portable calibre 4.99 w/ py3 branch source.)

In Python3.7, zipfile.py wants the file-like objects it is handed to have attribute seekable.

SpooledTemporaryFile doesn't always have that attribute.

Calibre's db.format(..., as_file=True) hands out a SpooledTemporaryFile (child class). Which I can't read as a zip file, rather defeating the purpose of asking for an epub, for example.

There is an Explanation about why SpooledTemporaryFile isn't seekable.

So the question is, should Calibre return something else for db.format(..., as_file=True), or should plugins not call that when they want a usable zip file?

Method db.cover(..., as_file=True) behaves the same. I haven't looked for other uses of SpooledTemporaryFile in Calibre.
JimmXinu is offline   Reply With Quote
Old 12-17-2019, 06:57 PM   #19
thiago.eec
Addict
thiago.eec knows who John Galt is.thiago.eec knows who John Galt is.thiago.eec knows who John Galt is.thiago.eec knows who John Galt is.thiago.eec knows who John Galt is.thiago.eec knows who John Galt is.thiago.eec knows who John Galt is.thiago.eec knows who John Galt is.thiago.eec knows who John Galt is.thiago.eec knows who John Galt is.thiago.eec knows who John Galt is.
 
Posts: 288
Karma: 71602
Join Date: Dec 2016
Location: Goiânia - Brazil
Device: iPad, Kindle Paperwhite
Quote:
Originally Posted by eschwartz View Post
@thiago.eec

You can get a debug log by using:

Preferences -> Restart in debug mode

It will create a text file containing the debug log, IIRC.
I did the porting using modernize now. Got better results:

1) ACE ported almost flawlessly. It works fine but for one error in the config dialog:
a) OPEN FOLDER. This gives the same error reported by @JimmXinu.
Spoiler:

calibre, version 4.99.0
ERRO: Exceção sem tratamento: <b>TypeError</b>:%b requires a bytes-like object, or an object that implements __bytes__, not 'int'

calibre 4.99 Portable embedded-python: True is64bit: False
Windows-10-10.0.18362-SP0 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.18362')
Python 3.8.0
Windows: ('10', '10.0.18362', 'SP0', '')
Interface language: pt_BR
Successfully initialized third party plugins: ACE (1, 1, 0) && Skoob Sync (0, 3, 0)
Traceback (most recent call last):
File "calibre_plugins.ACE.config", line 158, in get_directory
File "win_file_dialogs.py", line 238, in choose_dir
File "win_file_dialogs.py", line 161, in run_file_dialog
File "win_file_dialogs.py", line 59, in serialize_binary
TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'int'

2) Skoob Sync had more problems. First I had a problem with encoding a password with base64 (which I ended up giving up). There were a lot of other errors about encode/decode of strings/bytes with urllib requests.
Right now, it is working partially. Some functions work, others make calibre hang. Trying to debug now...

______________________________
EDIT: Looks like the problem is on my multi-threading code. Until now (4.6), sometimes I got an error like this:
Code:
QObject::setParent: Cannot set parent, new parent is in a different thread
But it did not prevent the plugin to run. Now, every time this error shows up, calibre hangs. Apparently, python 3 is more error intolerant ....

This is my code for multi-threading:
Spoiler:
Code:
    def run_parallel_in_threads(self, target, args_list):
        ''' Utility - spawn a thread to execute target for each args '''

        result = six.moves.queue.Queue()

        # Wrapper to collect return value in a Queue
        def task_wrapper(*args):
            result.put(target(*args))

        threads = [threading.Thread(target=task_wrapper, args=args) for args in args_list]

        count = 0

        for t in threads:
            count += 1
            t.start()
            # Don't send all requests at once
            list_size = len(args_list)
            if (list_size - 20) > 10:
                if count == 20:
                    time.sleep(1)
                elif count == 30:
                    time.sleep(1)
                elif count == 40:
                    time.sleep(2)
            elif (list_size - 20) > 0:
                if count == 20:
                    time.sleep(1)
            time.sleep(0.1)
            QApplication.processEvents(QtCore.QEventLoop.ExcludeUserInputEvents)
        for t in threads:
            t.join()
        return result

Anyway... I will keep digging.

Last edited by thiago.eec; 12-17-2019 at 07:41 PM.
thiago.eec is offline   Reply With Quote
Old 12-17-2019, 09:12 PM   #20
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,358
Karma: 83238367
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
@JimmXinu, @thiago.eec

Can you try out my fork here? https://github.com/eli-schwartz/calibre/commits/py3

I think this commit should fix the file dialogs error: https://github.com/eli-schwartz/cali...381ecdb581d274
eschwartz is offline   Reply With Quote
Advert
Old 12-17-2019, 09:13 PM   #21
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,358
Karma: 83238367
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
@jhowell,

Nothing in calibre depends on pycrypto, so I'm surprised you have it available in the prebuilt binaries at all.
eschwartz is offline   Reply With Quote
Old 12-17-2019, 09:31 PM   #22
JimmXinu
Plugin Developer
JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.JimmXinu ought to be getting tired of karma fortunes by now.
 
JimmXinu's Avatar
 
Posts: 3,917
Karma: 637184
Join Date: Dec 2011
Location: Midwest USA
Device: Kindle Voyage
Quote:
Originally Posted by eschwartz View Post
Can you try out my fork here? https://github.com/eli-schwartz/calibre/commits/py3

I think this commit should fix the file dialogs error: https://github.com/eli-schwartz/cali...381ecdb581d274
Working for me.
JimmXinu is offline   Reply With Quote
Old 12-17-2019, 09:39 PM   #23
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,358
Karma: 83238367
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
PR submitted as https://github.com/kovidgoyal/calibre/pull/1078
eschwartz is offline   Reply With Quote
Old 12-17-2019, 09:46 PM   #24
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,358
Karma: 83238367
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Quote:
Originally Posted by thiago.eec View Post
2) Skoob Sync had more problems. First I had a problem with encoding a password with base64 (which I ended up giving up). There were a lot of other errors about encode/decode of strings/bytes with urllib requests.
Indeed, the majority of issues you're likely to have are related to encoding/decoding strings, and tools like modernize/futurize usually cannot help with that. You'll just need to resolve them via (usually) simple trial and error.
eschwartz is offline   Reply With Quote
Old 12-17-2019, 10:09 PM   #25
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 36,093
Karma: 15014853
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
@jhowell: pycrypto is no longer maintained, so it has been replaced by pycryptodome, which is mostly a drop in replacement. https://github.com/Legrandin/pycryptodome
kovidgoyal is offline   Reply With Quote
Old 12-17-2019, 10:13 PM   #26
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,358
Karma: 83238367
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Quote:
Originally Posted by kovidgoyal View Post
@jhowell: pycrypto is no longer maintained, so it has been replaced by pycryptodome, which is mostly a drop in replacement. https://github.com/Legrandin/pycryptodome
Yeah, but the imports are really the same, and it's the import that fails.

https://github.com/kovidgoyal/calibr...rces.json#L694

Why does calibre even bundle this -- I don't have it as a dependency for calibre, and it's always worked just fine.

If I do extensive grepping of the sources, I see old versions of calibre bundled ezPyCrypto.py, but did not AFAICT actually use it.
eschwartz is offline   Reply With Quote
Old 12-17-2019, 10:33 PM   #27
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 36,093
Karma: 15014853
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by eschwartz View Post
Yeah, but the imports are really the same, and it's the import that fails.

Likely something in the windows freezing process, will tke a look at it whenI have a moment.
Quote:
Why does calibre even bundle this -- I don't have it as a dependency for calibre, and it's always worked just fine.

If I do extensive grepping of the sources, I see old versions of calibre bundled ezPyCrypto.py, but did not AFAICT actually use it.
Because there was a time when it did bundle it and various plugins started to depend on it. calibre itself no longer uses it.
kovidgoyal is offline   Reply With Quote
Old 12-17-2019, 11:13 PM   #28
jhowell
Wizard
jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.jhowell ought to be getting tired of karma fortunes by now.
 
jhowell's Avatar
 
Posts: 3,369
Karma: 32493247
Join Date: Nov 2011
Location: Tampa Bay, Florida
Device: Oasis 2, Fire, iPad Air 2, Nexus 7
Quote:
Originally Posted by eschwartz View Post
Yeah, but the imports are really the same, and it's the import that fails.
From the error message it appears that there is a missing file causing the failure:
_raw_ecb.cp38-win32.pyd


Quote:
Originally Posted by eschwartz View Post
Why does calibre even bundle this -- I don't have it as a dependency for calibre, and it's always worked just fine.
Cyptographic functions are useful. I use them in some of my personal plugins and they are also used by a popular plugin that is not distributed via MobileRead.
jhowell is online now   Reply With Quote
Old 12-17-2019, 11:25 PM   #29
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 36,093
Karma: 15014853
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by JimmXinu View Post

SpooledTemporaryFile doesn't always have that attribute.
Ugh this is just python devs being lazy: https://bugs.python.org/issue26175

Guess I will have to do their work for them.
kovidgoyal is offline   Reply With Quote
Old 12-18-2019, 12:43 AM   #30
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 36,093
Karma: 15014853
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
I uploaded 4.99.1 with fixes for all reported issues.
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Plugin Devs: Should we include a License or COPYING file? KevinH Plugins 3 08-15-2017 02:09 PM
Attn plugin developers: calibre moving to Qt 5 kovidgoyal Plugins 157 01-02-2015 12:22 AM
Can I develop a plugin in a launguage other than Python? jamawg Development 2 06-13-2014 01:38 AM
Calibre in a Python 3.2 world, attn: Kovid Kevin McAleavey Calibre 8 01-09-2012 06:49 PM
How do I Create a Python Plugin? Sydney's Mom Plugins 25 01-27-2010 07:26 AM


All times are GMT -4. The time now is 11:39 AM.


MobileRead.com is a privately owned, operated and funded community.