Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 10-07-2020, 12:45 PM   #1
bgme
Junior Member
bgme began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Oct 2017
Device: kindle
[Metadata Source Plugin] Douban PY3 updated

This plugin can fetch metadata from 豆瓣读书.
This plugin is removed from core source on this commit.


Version 3.2.0 - 8 Oct 2020
Fix: Changes for Python 3 support in calibre.
Attached Files
File Type: zip douban.zip (3.6 KB, 18681 views)
bgme is offline   Reply With Quote
Old 03-20-2021, 08:28 PM   #2
likekindle
Member
likekindle began at the beginning.
 
Posts: 21
Karma: 10
Join Date: Apr 2019
Device: Kindle
Hi, this plugin is down, maybe douban blocked it, do you have plan to fix it?
likekindle is offline   Reply With Quote
Advert
Old 03-26-2021, 08:33 PM   #3
kyalpha
Junior Member
kyalpha began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Mar 2021
Device: kindle
Quote:
Originally Posted by likekindle View Post
Hi, this plugin is down, maybe douban blocked it, do you have plan to fix it?
Error Message

calibre, version 5.14.0
ERROR: No matches found: <p>Failed to find any books that match your search. Try making the search <b>less specific</b>. For example, use only the author's last name and a single distinctive word from the title.<p>To see the full log, click "Show details".

Running identify query with parameters:
{'title': '龙眠', 'authors': ['宫部美雪'], 'identifiers': {'mobi-asin': '8a4ea100-c28b-4d9a-9ef8-f2b3a7cb253c'}, 'timeout': 30}
Using plugins: Douban Books (3, 2, 0)
The log from individual plugins is below

****************************** Douban Books (3, 2, 0) ******************************
Found 0 results
Downloading from Douban Books took 0.13665342330932617
Failed to make identify query: 'https://api.douban.com/v2/book/search?count=10&q=%E9%BE%99%E7%9C%A0+%E5%AE%AB%E9% 83%A8%E7%BE%8E%E9%9B%AA&apikey=054022eaeae0b00e0fc 068c0c0a2102a'
Traceback (most recent call last):
File "calibre_plugins.dummy1.__init__", line 334, in identify
File "mechanize\_mechanize.py", line 241, in open_novisit
File "mechanize\_mechanize.py", line 313, in _mech_open
mechanize._response.get_seek_wrapper_class.<locals >.httperror_seek_wrapper: HTTP Error 403: Forbidden

************************************************** ******************************
The identify phase took 0.20 seconds
The longest time (0.136653) was taken by: Douban Books
Merging results from different sources
We have 0 merged results, merging took: 0.00 seconds
kyalpha is offline   Reply With Quote
Old 03-28-2021, 09:28 AM   #4
likekindle
Member
likekindle began at the beginning.
 
Posts: 21
Karma: 10
Join Date: Apr 2019
Device: Kindle
Quote:
Originally Posted by kyalpha View Post
Error Message

calibre, version 5.14.0
ERROR: No matches found: <p>Failed to find any books that match your search. Try making the search <b>less specific</b>. For example, use only the author's last name and a single distinctive word from the title.<p>To see the full log, click "Show details".

Running identify query with parameters:
{'title': '龙眠', 'authors': ['宫部美雪'], 'identifiers': {'mobi-asin': '8a4ea100-c28b-4d9a-9ef8-f2b3a7cb253c'}, 'timeout': 30}
Using plugins: Douban Books (3, 2, 0)
The log from individual plugins is below

****************************** Douban Books (3, 2, 0) ******************************
Found 0 results
Downloading from Douban Books took 0.13665342330932617
Failed to make identify query: 'https://api.douban.com/v2/book/search?count=10&q=%E9%BE%99%E7%9C%A0+%E5%AE%AB%E9% 83%A8%E7%BE%8E%E9%9B%AA&apikey=054022eaeae0b00e0fc 068c0c0a2102a'
Traceback (most recent call last):
File "calibre_plugins.dummy1.__init__", line 334, in identify
File "mechanize\_mechanize.py", line 241, in open_novisit
File "mechanize\_mechanize.py", line 313, in _mech_open
mechanize._response.get_seek_wrapper_class.<locals >.httperror_seek_wrapper: HTTP Error 403: Forbidden

************************************************** ******************************
The identify phase took 0.20 seconds
The longest time (0.136653) was taken by: Douban Books
Merging results from different sources
We have 0 merged results, merging took: 0.00 seconds
Hi, how do you get this log message?
likekindle is offline   Reply With Quote
Old 03-28-2021, 07:49 PM   #5
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 22,606
Karma: 40194119
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by likekindle View Post
Hi, how do you get this log message?
When the results of the metadata download are displayed for you to choose the book, there is a button for viewing the log. Pressing that will show what @kyalpha has posted.

But, based on the error displayed, "HTTP Error 403: Forbidden" and visiting the search URL in the message, the API key used by the plugin is no longer valid. That could be because it has expired or because the use broke the usage rules. Someone needs to renew it.
davidfor is online now   Reply With Quote
Advert
Old 03-31-2021, 08:58 PM   #6
likekindle
Member
likekindle began at the beginning.
 
Posts: 21
Karma: 10
Join Date: Apr 2019
Device: Kindle
Quote:
Originally Posted by davidfor View Post
When the results of the metadata download are displayed for you to choose the book, there is a button for viewing the log. Pressing that will show what @kyalpha has posted.

But, based on the error displayed, "HTTP Error 403: Forbidden" and visiting the search URL in the message, the API key used by the plugin is no longer valid. That could be because it has expired or because the use broke the usage rules. Someone needs to renew it.
The message said:
Failed to download the metadata or cover for the selected book.
Unknown (Failed metadata)

By this message, I cannot get details why it failed.
(Sorry i cannot upload the screenshot cause the "Insert image" button is not working on my Chrome/Edge)
likekindle is offline   Reply With Quote
Old 03-31-2021, 10:42 PM   #7
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 22,606
Karma: 40194119
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by likekindle View Post
The message said:
Failed to download the metadata or cover for the selected book.
Unknown (Failed metadata)

By this message, I cannot get details why it failed.
(Sorry i cannot upload the screenshot cause the "Insert image" button is not working on my Chrome/Edge)
The "View log" button will be displayed if there are any results. But, if there are no results and error message dialog is displayed, the buttons will be different. The "Details" will show the log in the same way as the "View log" button. And the "Copy to clipboard" will copy those contents to the clipboard and you can paste it into post.
davidfor is online now   Reply With Quote
Old 04-02-2021, 01:09 AM   #8
likekindle
Member
likekindle began at the beginning.
 
Posts: 21
Karma: 10
Join Date: Apr 2019
Device: Kindle
calibre, version 5.10.1
ERROR: Download failed: Failed to download metadata or cover for the selected book.

未知 (Failed metadata)

this is the message i copied, also no details, maybe the plugin not write the error log.
likekindle is offline   Reply With Quote
Old 04-03-2021, 09:15 AM   #9
likekindle
Member
likekindle began at the beginning.
 
Posts: 21
Karma: 10
Join Date: Apr 2019
Device: Kindle
hi, i've made a new version of this plugin, but it's not working for now,
when i install it, the error message pop up:

Quote:
calibre 5.14* [64bit] embedded-python: True is64bit: True
Windows-10-10.0.19041 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: zh_CN
Traceback (most recent call last):
File "D:\code\calibre-5.14.0\src\calibre\gui2\preferences\plugins.py", line 316, in add_plugin
File "D:\code\calibre-5.14.0\src\calibre\customize\ui.py", line 472, in add_plugin
File "D:\code\calibre-5.14.0\src\calibre\customize\ui.py", line 61, in load_plugin
File "D:\code\calibre-5.14.0\src\calibre\customize\zipplugin.py", line 293, in load
File "importlib\__init__.py", line 127, in import_module
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
File "D:\code\calibre-5.14.0\src\calibre\customize\zipplugin.py", line 192, in exec_module
File "calibre_plugins.dummy1.__init__", line 23, in <module>
ModuleNotFoundError: No module named 'pandas'
i used pandas and pymongo in the plugin, but i've installed them using pip, why it still failed to run?

PS. this version is based on Li Fanxi's work, for more details, plz check the code.
PPS. this is the plugin file as attachment, but I'm not sure it's available for you.
https://www.mobileread.com/forums/at...1&d=1617454991
Attached Files
File Type: zip douban.zip (4.1 KB, 237 views)
likekindle is offline   Reply With Quote
Old 04-03-2021, 09:50 PM   #10
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 22,606
Karma: 40194119
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by likekindle View Post
hi, i've made a new version of this plugin, but it's not working for now,
when i install it, the error message pop up:



i used pandas and pymongo in the plugin, but i've installed them using pip, why it still failed to run?

PS. this version is based on Li Fanxi's work, for more details, plz check the code.
PPS. this is the plugin file as attachment, but I'm not sure it's available for you.
https://www.mobileread.com/forums/at...1&d=1617454991
Calibre includes its own Python installation. It does not use anything from the Python installed in the OS. If you want to use modules not included with calibre, you need to package them yourself.

But, how are you using Pandas and PyMongo? I would have thought Pandas was overkill. But, I haven't touched it enough to see what advantage it would have over other methods. But, you completely baffle me with PyMongo. As it is an interface for MongoDB, I could only see it being useful if Douban is allowing direct access to their database.
davidfor is online now   Reply With Quote
Old 04-04-2021, 12:44 AM   #11
likekindle
Member
likekindle began at the beginning.
 
Posts: 21
Karma: 10
Join Date: Apr 2019
Device: Kindle
Quote:
Originally Posted by davidfor View Post
Calibre includes its own Python installation. It does not use anything from the Python installed in the OS. If you want to use modules not included with calibre, you need to package them yourself.

But, how are you using Pandas and PyMongo? I would have thought Pandas was overkill. But, I haven't touched it enough to see what advantage it would have over other methods. But, you completely baffle me with PyMongo. As it is an interface for MongoDB, I could only see it being useful if Douban is allowing direct access to their database.
Yes, you're right, the pandas/pymongo is not used in the plugin(maybe i copied the code from somewhere), now i removed the dependencies, then the plugin is installed(using command "calibre-customize -b ." BTW)
But when i start the app using command "calibre-debug -g", then use the plugin, it's still not working, the error message is like:
Quote:
calibre, version 5.14.0
ERROR: Download failed: Failed to download metadata or cover for the selected book.

Unknown (Failed metadata)
I've write some log message(using print statement), but they are not showed. i don't know why, and how to debug the plugin?

Thanks a lot for help.
likekindle is offline   Reply With Quote
Old 04-04-2021, 01:44 AM   #12
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 22,606
Karma: 40194119
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Looking at the code you posted, you are using print statements rather than writing to the log. The log is a parameter of the identify method. Writing anything to that will appear in the log that calibre displays.

And I cannot install the version you posted after removing the extra imports. The problem is the __init__ does not have the correct parameters. And there is no call to the parent class to initialize it. It isn't really needed. You should set those properties at the class level, or as constants.
davidfor is online now   Reply With Quote
Old 04-04-2021, 07:52 AM   #13
likekindle
Member
likekindle began at the beginning.
 
Posts: 21
Karma: 10
Join Date: Apr 2019
Device: Kindle
Just as you said, the __init__ is not needed, so i removed it, then use log to write log message.
But it still not working, I'll post the new code, and error message i got, could you see what's wrong in it?

Quote:

Hello, calibre!
PS D:\douban> calibre-customize -b .
Hello, calibre!
Plugin updated: Douban Books v2 (4, 2, 1)
PS D:\douban> calibre-debug -g
Hello, calibre!
calibre 5.14* [64bit] embedded-python: True is64bit: True
Windows-10-10.0.19041 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: Douban Books v2 (4, 2, 1)
Turning on automatic hidpi scaling
devicePixelRatio: 1.0
logicalDpi: 120.0 x 120.0
physicalDpi: 157.8252427184466 x 157.65517241379308
Using calibre Qt style: True
[0.00] Starting up...
[0.00] Showing splash screen...
[0.47] splash screen shown
[0.47] Initializing db...
[0.47] db initialized
[0.47] Constructing main UI...
[1.62] main UI initialized...
[1.62] Hiding splash screen
Starting QuickView
[1.80] splash screen hidden
[1.80] Started up in 1.80 seconds with 3 books
Metadata sources cache was recently updated not updating again
Job: 0 Download metadata for one book finished
Starting job: Download metadata for one book
Download complete, with 1 failures
Attached Files
File Type: py __init__.py (13.8 KB, 93 views)
likekindle is offline   Reply With Quote
Old 04-04-2021, 09:19 AM   #14
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 22,606
Karma: 40194119
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
From that, you are using the batch metadata download. Don't. Do your debugging from the metadata editor. That will show the error and the log will show the full details of the attempt to download the metadata and any errors. That is what @kyalpha used and what I was referring to.

Using your code for a random book in my library shows:

Code:
calibre, version 5.14.0
ERROR: No matches found: <p>Failed to find any books that match your search. Try making the search <b>less specific</b>. For example, use only the author's last name and a single distinctive word from the title.<p>To see the full log, click "Show details".

Running identify query with parameters: 
{'title': 'Alien Hunters', 'authors': ['Daniel Arenson'], 'identifiers': {'isbn': '9781505658200', 'amazon': '1505658209', 'ff': 'a/daniel-arenson/alien-hunters', 'fictiondb': 'daniel-arenson~alien-hunters~1197738~b', 'goodreads': '24533763', 'kobo': 'alien-hunters'}, 'timeout': 30} 
Using plugins: Douban Books v2 (4, 2, 1) 
The log from individual plugins is below 

****************************** Douban Books v2 (4, 2, 1) ****************************** 
Found 0 results 
Downloading from Douban Books v2 took 4.281980037689209 
start get metadata from douban...
{'isbn': '9781505658200', 'amazon': '1505658209', 'ff': 'a/daniel-arenson/alien-hunters', 'fictiondb': 'daniel-arenson~alien-hunters~1197738~b', 'goodreads': '24533763', 'kobo': 'alien-hunters'}
9781505658200
http://douban.com/isbn/9781505658200
Plugin Douban Books v2 failed
Traceback (most recent call last):
  File "E:\Development\GitHub\calibre\src\calibre\ebooks\metadata\sources\identify.py", line 47, in run
  File "calibre_plugins.dummy2.__init__", line 91, in identify
  File "calibre_plugins.dummy2.__init__", line 292, in get_book
  File "calibre_plugins.dummy2.__init__", line 377, in __get_author
IndexError: list index out of range 

******************************************************************************** 
The identify phase took 4.30 seconds 
The longest time (4.281980) was taken by: Douban Books v2 
Merging results from different sources 
We have 0 merged results, merging took: 0.00 seconds
davidfor is online now   Reply With Quote
Old 04-04-2021, 09:36 AM   #15
likekindle
Member
likekindle began at the beginning.
 
Posts: 21
Karma: 10
Join Date: Apr 2019
Device: Kindle
Quote:
Do your debugging from the metadata editor. That will show the error and the log will show the full details of the attempt to download the metadata and any errors.
sorry, i don't know how to do that...

BTW, i've fixed the bugs, now it's working~~~
thanks for all your help. and i'll do some more tests on it, then i could post it here.
likekindle is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[Metadata Source Plugin] DatabazeKnih (cz) baghira Plugins 203 10-08-2021 03:00 AM
[Metadata Source Plugin] LubimyCzytac [PL] UPDATED BeckyEbook Plugins 77 09-21-2021 07:28 AM
[Metadata Source Plugin] FictionDB kiwidude Plugins 100 03-07-2021 07:20 PM
[Metadata Source Plugin] Empty Plugin? (Fake Identifier) mneimeyer Plugins 3 11-11-2019 08:07 PM
[Metadata Source Plugin] BIBLIOETECA biblioeteca Plugins 12 10-28-2015 12:39 PM


All times are GMT -4. The time now is 09:27 AM.


MobileRead.com is a privately owned, operated and funded community.