[GUI Plugin] WordDumb - Page 9

xxyzz · 05-17-2021, 08:21 PM

Quote:

Originally Posted by thenextstar

I am getting the following error on M1 Macbook Air.

I forgot mac's builtin pip3 needs root permission to install packages, v3.7.0 should fix this issue.

v3.7.0:
- add 943 lemmas and update difficulty level of some lemmas
- don't update pip before installing nltk
- check installed numpy python version
- use book text for x-ray that get Wikipedia disambiguation result
- check redirects and normalized page titles from Wikipedia API
- delete nltk model zip files

mallomar · 05-18-2021, 03:52 PM

Did something change that it should work on Mac now? I'm still getting the error thenextstar mentioned, even after u[dating to the latest version.

xxyzz · 05-18-2021, 08:26 PM

Quote:

Originally Posted by mallomar

Did something change that it should work on Mac now? I'm still getting the error thenextstar mentioned, even after u[dating to the latest version.

The new version doesn't run the `pip install -U` command like the error message thenextstar posted. What's your error message?

Mac's pip3 is too old, I'd recommend use Homebrew's python or update the built-in pip:

Code:

$ sudo -H /usr/bin/python3 -m pip install -U pip

mallomar · 05-19-2021, 03:46 PM

Quote:

Originally Posted by xxyzz

The new version doesn't run the `pip install -U` command like the error message thenextstar posted. What's your error message?

Code:

calibre, version 5.17.0 (darwin, embedded-python: True)
Dumb error: Failed: Generating Word Wise for The Largesse of the Sea Maiden

Starting job: Generating Word Wise for The Largesse of the Sea Maiden 
Job: "Generating Word Wise for The Largesse of the Sea Maiden" failed with error: 
Traceback (most recent call last):
  File "calibre/gui2/threaded_jobs.py", line 83, in start_work
  File "calibre_plugins.worddumb.parse_job", line 17, in do_job
    install_libs()
  File "calibre_plugins.worddumb.unzip", line 26, in install_libs
    pip_install('nltk', NLTK_VERSION)
  File "calibre_plugins.worddumb.unzip", line 83, in pip_install
    subprocess.check_call(
  File "subprocess.py", line 364, in check_call
subprocess.CalledProcessError: Command '['/usr/bin/pip3', 'install', '-t', PosixPath('/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/nltk3.6.2'), 'nltk==3.6.2']' returned non-zero exit status 1.
 
Called with args: ((84, 'KFX', 'B0722TJSG7', '/Users/rapaz/Calibre Library/Denis Johnson/The Largesse of the Sea Maiden (84)/The Largesse of the Sea Maiden - Denis Johnson.kfx', <calibre.ebooks.metadata.book.base.Metadata object at 0x13475a160>), True) {'notifications': <queue.Queue object at 0x1350e60d0>, 'abort': <threading.Event object at 0x1350e6550>, 'log': <calibre.utils.logging.GUILog object at 0x1350e62b0>}

Quote:

Originally Posted by xxyzz

Mac's pip3 is too old, I'd recommend use Homebrew's python or update the built-in pip:

Code:

$ sudo -H /usr/bin/python3 -m pip install -U pip

I tried putting this command in Terminal but it says:

Code:

xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun

xxyzz · 05-19-2021, 07:44 PM

Quote:

Originally Posted by mallomar

I tried putting this command in Terminal but it says:

Code:

xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun

Please post the output of these commands:

Code:

$ sw_vers -productVersion
$ /usr/bin/pip3 --version
$ /usr/bin/pip3 install -t ~/Library/Preferences/calibre/plugins/worddumb-libs/nltk3.6.2 nltk==3.6.2

Are you using a m1 mac?

The last error means you need to install developer tools:

Code:

$ xcode-select --install

Maybe mac's outdated pip wants to build the 'regex' package but can't find any required build tools, install the developer tool then update pip should fix your problem.

You can also install python via Homebrew.

mallomar · 05-20-2021, 05:38 PM

[QUOTE=xxyzz;4123008]Please post the output of these commands:

Code:

$ sw_vers -productVersion
$ /usr/bin/pip3 --version
$ /usr/bin/pip3 install -t ~/Library/Preferences/calibre/plugins/worddumb-libs/nltk3.6.2 nltk==3.6.2

Code:

~ % sw_vers -productVersion
 /usr/bin/pip3 --version
 /usr/bin/pip3 install -t ~/Library/Preferences/calibre/plugins/worddumb-libs/nltk3.6.2 nltk==3.6.2
11.4
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
pip 21.1.1 from /Library/Python/3.8/site-packages/pip (python 3.8)
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Collecting nltk==3.6.2
  Downloading nltk-3.6.2-py3-none-any.whl (1.5 MB)
     |████████████████████████████████| 1.5 MB 3.0 MB/s 
Collecting joblib
  Downloading joblib-1.0.1-py3-none-any.whl (303 kB)
     |████████████████████████████████| 303 kB 2.4 MB/s 
Collecting click
  Downloading click-8.0.1-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 2.0 MB/s 
Collecting tqdm
  Downloading tqdm-4.60.0-py2.py3-none-any.whl (75 kB)
     |████████████████████████████████| 75 kB 3.7 MB/s 
Collecting regex
  Downloading regex-2021.4.4-cp38-cp38-macosx_10_9_x86_64.whl (284 kB)
     |████████████████████████████████| 284 kB 3.2 MB/s 
Installing collected packages: tqdm, regex, joblib, click, nltk
Successfully installed click-8.0.1 joblib-1.0.1 nltk-3.6.2 regex-2021.4.4 tqdm-4.60.0

It seems like it did something as I no longer get an error in Calibre with Wordumb, but it also seems to do nothing now.

Quote:

Originally Posted by xxyzz

Are you using a m1 mac?

No, Intel.

Quote:

Originally Posted by xxyzz

The last error means you need to install developer tools:

Code:

$ xcode-select --install

Maybe mac's outdated pip wants to build the 'regex' package but can't find any required build tools, install the developer tool then update pip should fix your problem.

You can also install python via Homebrew.

Installed succesfully.

xxyzz · 05-20-2021, 11:40 PM

v3.7.2

- run pip in `python -m pip`
- fix Homebrew python path on ARM macOS
- use calibre's MOBI MetadataUpdater to get and set ASIN, requires calibre 5.18.0

xxyzz · 05-21-2021, 09:38 PM

v3.7.3

- install nltk and dependencies explicitly to avoid building regex for outdated pip on macOS
- use Python Launcher on Windows

Use sudo to update mac's built-in pip is not a good idea...

mallomar · 05-22-2021, 12:04 PM

I'm still getting an error on the latest build BUT it actually seems like it's doing something for once, so that's improvements. Do I still need to be disabling gatekeeper?

Code:

calibre, version 5.18.0 (darwin, embedded-python: True)
Dumb error: Failed: Generating Word Wise for The Largesse of the Sea Maiden

Starting job: Generating Word Wise for The Largesse of the Sea Maiden 
Job: "Generating Word Wise for The Largesse of the Sea Maiden" failed with error: 
Traceback (most recent call last):
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/__init__.py", line 22, in <module>
    from . import multiarray
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: dlopen(/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so, 2): no suitable image found.  Did find:
	/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so: code signature in (/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre/gui2/threaded_jobs.py", line 83, in start_work
  File "calibre_plugins.worddumb.parse_job", line 32, in do_job
    find_named_entity(start, text, x_ray)
  File "calibre_plugins.worddumb.parse_job", line 98, in find_named_entity
    nodes = ne_chunk(pos_tag(word_tokenize(text)))
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/nltk/nltk/chunk/__init__.py", line 185, in ne_chunk
    chunker = load(chunker_pickle)
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/nltk/nltk/data.py", line 755, in load
    resource_val = pickle.load(opened_resource)
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/__init__.py", line 145, in <module>
    from . import core
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/Applications/calibre.app/Contents/MacOS/calibre"
  * The NumPy version is: "1.20.3"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: dlopen(/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so, 2): no suitable image found.  Did find:
	/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so: code signature in (/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

 
Called with args: ((461, 'KFX', 'B0722TJSG7', '/Users/rapaz/Calibre Library/Denis Johnson/The Largesse of the Sea Maiden (461)/The Largesse of the Sea Maiden - Denis Johnson.kfx', <calibre.ebooks.metadata.book.base.Metadata object at 0x128556c10>), True) {'notifications': <queue.Queue object at 0x11120e100>, 'abort': <threading.Event object at 0x127321130>, 'log': <calibre.utils.logging.GUILog object at 0x127321250>}

xxyzz · 05-22-2021, 07:20 PM

Quote:

Originally Posted by mallomar

I'm still getting an error on the latest build BUT it actually seems like it's doing something for once, so that's improvements. Do I still need to be disabling gatekeeper?

X-ray is disabled by default on mac, did you forget to disable it?

mallomar · 05-28-2021, 12:03 PM

Quote:

Originally Posted by xxyzz

X-ray is disabled by default on mac, did you forget to disable it?

It's enabled. I'm still getting the same error, though oddly on other books when I try it nothing seems to happen at all.

Code:

calibre, version 5.19.0 (darwin, embedded-python: True)
Dumb error: Failed: Generating Word Wise for The Largesse of the Sea Maiden

Starting job: Generating Word Wise for The Largesse of the Sea Maiden 
Job: "Generating Word Wise for The Largesse of the Sea Maiden" failed with error: 
Traceback (most recent call last):
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/__init__.py", line 22, in <module>
    from . import multiarray
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/multiarray.py", line 12, in <module>
    from . import overrides
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/overrides.py", line 7, in <module>
    from numpy.core._multiarray_umath import (
ImportError: dlopen(/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so, 2): no suitable image found.  Did find:
	/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so: code signature in (/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre/gui2/threaded_jobs.py", line 83, in start_work
  File "calibre_plugins.worddumb.parse_job", line 32, in do_job
    find_named_entity(start, text, x_ray)
  File "calibre_plugins.worddumb.parse_job", line 98, in find_named_entity
    nodes = ne_chunk(pos_tag(word_tokenize(text)))
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/nltk/nltk/chunk/__init__.py", line 185, in ne_chunk
    chunker = load(chunker_pickle)
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/nltk/nltk/data.py", line 755, in load
    resource_val = pickle.load(opened_resource)
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/__init__.py", line 145, in <module>
    from . import core
  File "/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/__init__.py", line 48, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/Applications/calibre.app/Contents/MacOS/calibre"
  * The NumPy version is: "1.20.3"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: dlopen(/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so, 2): no suitable image found.  Did find:
	/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so: code signature in (/Users/rapaz/Library/Preferences/calibre/plugins/worddumb-libs/numpy_38/numpy/core/_multiarray_umath.cpython-38-darwin.so) not valid for use in process using Library Validation: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

 
Called with args: ((461, 'KFX', 'B0722TJSG7', '/Users/rapaz/Calibre Library/Denis Johnson/The Largesse of the Sea Maiden (461)/The Largesse of the Sea Maiden - Denis Johnson.kfx', <calibre.ebooks.metadata.book.base.Metadata object at 0x129057790>), True) {'notifications': <queue.Queue object at 0x129068880>, 'abort': <threading.Event object at 0x129068430>, 'log': <calibre.utils.logging.GUILog object at 0x129068490>}

xxyzz · 05-28-2021, 07:29 PM

Quote:

Originally Posted by mallomar

It's enabled. I'm still getting the same error, though oddly on other books when I try it nothing seems to happen at all.

Disable x-ray.

This plugin won't do anything on non-English eBooks(press 'i' to check the book metadata) and not supported eBooks(like EPUB).

devil2trap · 05-31-2021, 12:33 PM

First of all thanks for this beautiful plugin. But this plugin is giving haywire results for wordwise when wifi is kept "On".With Airplane mode "On" there are no issues. And I tried literally everything but cannot figure out how to make this plugin work with keeping wifi on.

The procedure of my conversion of books and sending to Kindle:
1)Epub to Kfx
2)Edit Metadata & Covers
3)Using Worddumb plugin to send books to kindle.

Also I have tried ur earlier response to remove "mobi-asin" but it didn't solve the issue.

So, if you could help me out I'll really appreciate that.

devil2trap · 05-31-2021, 01:38 PM

Quote:

Originally Posted by astro70.1

Wow it works out perfect! Really appreciate your quick apply and kindness. Probably for someone who wants to register downloaded book to amazon or goodreads they can just transfer it without using plugin, so I think it won't be any problem. Anyway thanks for your great work!

Bro how did you make it work because my wordwise always go haywire when I switch my wifi on... Please help me out.

xxyzz · 05-31-2021, 11:08 PM

Quote:

Originally Posted by devil2trap

2)Edit Metadata & Covers

Delete the KFX book, convert again but this time don't change the ASIN. I don't know the details of the step one and two but that ASIN doesn't come from nowhere. You might need to delete the mobi-asin metadata before converting the book.

Don't use Amazon's ASIN on books you don't buy from kindle store then turn on wifi and expect they don't know about it...

05-31-2021, 12:33 PM	#133
devil2trap Member Posts: 11 Karma: 10 Join Date: May 2021 Device: kindle paperwhite	Worddumb not working with WiFi kept on First of all thanks for this beautiful plugin. But this plugin is giving haywire results for wordwise when wifi is kept "On".With Airplane mode "On" there are no issues. And I tried literally everything but cannot figure out how to make this plugin work with keeping wifi on. The procedure of my conversion of books and sending to Kindle: 1)Epub to Kfx 2)Edit Metadata & Covers 3)Using Worddumb plugin to send books to kindle. Also I have tried ur earlier response to remove "mobi-asin" but it didn't solve the issue. So, if you could help me out I'll really appreciate that.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[GUI Plugin] Manga plugin	mastertea	Plugins	7	Today 12:13 AM
[GUI Plugin] KindleUnpack - The Plugin	DiapDealer	Plugins	527	08-15-2025 01:36 PM
[GUI Plugin] CalibreSpy	DaltonST	Plugins	245	08-18-2024 09:33 PM
[GUI Plugin] Save Virtual Libraries To Column (GUI)	chaley	Plugins	14	04-04-2021 05:25 AM
[GUI Plugin] Plugin Updater Deprecated	kiwidude	Plugins	159	06-19-2011 12:27 PM

05-18-2021, 03:52 PM	#122
mallomar Connoisseur Posts: 71 Karma: 8682 Join Date: May 2018 Device: Onyx Boox Note Air 2, Kindle Oasis 2	Did something change that it should work on Mac now? I'm still getting the error thenextstar mentioned, even after u[dating to the latest version.

05-20-2021, 11:40 PM	#127
xxyzz Evangelist Posts: 448 Karma: 3000000 Join Date: Nov 2020 Device: none	v3.7.2 - run pip in `python -m pip` - fix Homebrew python path on ARM macOS - use calibre's MOBI MetadataUpdater to get and set ASIN, requires calibre 5.18.0

05-21-2021, 09:38 PM	#128
xxyzz Evangelist Posts: 448 Karma: 3000000 Join Date: Nov 2020 Device: none	v3.7.3 - install nltk and dependencies explicitly to avoid building regex for outdated pip on macOS - use Python Launcher on Windows Use sudo to update mac's built-in pip is not a good idea...