Download Metadata - V4.x OK but NOT V5.x

[Rob557]

When my Kobo could not connect to Calibre V4.23, I belatedly upgraded within the last week to Calibre V5.5 (and then 5.6) and the most recent plug-ins.

HOWEVER, I am finding that the "download metadata" feature (available when right-click on multiple ePubs that are missing metadata) does not seem to work under V5.x. I can see the metadata when looking at "view log" but when I look at "review downloaded metadata" none of the information is shown and no update is made to the displayed metadata (i.e. comments, ratings, tags, published date, etc).

A temporary ad hoc solution that works is simply uninstalling Calibre V5.6, reinstalling Calibre V4.23, downloading/updating the metadata (the metadata is then viewable when click on "review downloaded metadata" and the database updates okay). I can then reinstall Calibre V5.6

But Calibre V5.x seems to require a fix for this feature, or is there some way in which I have to change the configuration I have under Calibre V5.x?

As noted, I have updated all the plugins that I use, but the "download metadata" feature that appears when right-click on ePubs seems to be an integral feature of Calibre and not a second-party plug-in.

I'm surprised others don't seem to have this systemic problem, although I noted there are some people who seem to have what they perceive as one-off problems with specific books??

Thanks,
Rob

[davidfor]

I haven't seen any issues with the metadata download and updating the books. I normally do this from the metadata editor, but, have used the bulk metadata download (the "Download metadata and covers" option) a bit recently. Any time metadata has been found, it is displayed, and if I accept it, the metadata is updated.

Probably the first thing to check is the metadata download configuration. This is in the preferences. Make sure that you have fields selected under "Metadata fields to download" that is in the top right of the configuration. And the same can be configured in the enabled metadata sources. These set which source supplies which field, and the main configuration sets which fields are updated at all.

Other than that, we need some examples. And the logs. Hopefully that will give a hint as to what is going wrong.

[Rob557]

Originally Posted by davidfor

Other than that, we need some examples. And the logs. Hopefully that will give a hint as to what is going wrong.

Hi David,

I've run two examples, but for simplicity I've included below detail only for Steve Berry:
1. Steve Berry - The Patriot Threat
2. Jim Gaffigan - Food: A Love Story

Below I've copied the metadata download log for Steve Berry and you can see it is picking up comments, tags, published date. I don't know how to insert an image of the "review downloaded metadata" display but the fields for tags, comments, etc are simply blank and no metadata is updated (under Calibre V5,x ... but shows contents and updates okay if run under Calbre 4.x).


#################### Log for The Patriot Threat ####################
Running identify query with parameters:
{'title': 'The Patriot Threat', 'authors': ['Steve Berry'], 'identifiers': {'goodreads': '22572547', 'barnesnoble': 'w/the-patriot-threat-steve-berry/1119182809', 'mobi-asin': 'B00L74ADYG', 'isbn': '9781466862609', 'amazon': 'B00L74ADYG', 'ff': 'b/steve-berry/patriot-threat', 'google': 'cTWdBAAAQBAJ'}, 'timeout': 30}
Using plugins: Google (1, 0, 1), Amazon.com (1, 2, 14)
The log from individual plugins is below

****************************** Google (1, 0, 1) ******************************
Found 1 results
Downloading from Google took 0.5989999771118164


---
Title : The Patriot Threat: A Novel
Author(s) : Steve Berry
Publisher : St. Martin's Publishing Group
Tags : Fiction, Thrillers, Espionage, Historical, Political
Languages : eng
Published : 2015-03-31T00:27:05.231877+00:00
Identifiers : google:cTWdBAAAQBAJ, isbn:9781466862609
Comments : Steve Berry's New York Times bestseller, The Patriot Threat, finds Cotton Malone racing to stop a rogue ex-KGB agent plotting revenge against the United States.The 16th Amendment to the Constitution is why Americans pay income taxes. But what if there were problems associated with that amendment? Secrets that call into question decades of tax collecting? In fact, there is a surprising truth to this hidden possibility.Cotton Malone, once a member of an elite intelligence division within the Justice Department known as the Magellan Billet, is now retired and owns an old bookshop in Denmark. But when his former-boss, Stephanie Nelle, asks him to track a rogue North Korean who may have acquired some top secret Treasury Department files—the kind that could bring the United States to its knees—Malone is vaulted into a harrowing twenty-four hour chase that begins on the canals in Venice and ends in the remote highlands of Croatia.With appearances by Franklin Roosevelt, Andrew Mellon, a curious painting that still hangs in the National Gallery of Art, and some eye-opening revelations from the $1 bill, this riveting, non-stop adventure is trademark Steve Berry—90% historical fact, 10% exciting speculation—a provocative thriller posing a dangerous question: What if the Federal income tax is illegal?
Making query: https://books.google.com/books/feeds...ewability=none

************************************************** ******************************

****************************** Amazon.com (1, 2, 14) ******************************
Found 0 results
Downloading from Amazon.com took 1.0109963417053223
User-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
Server: auto
Making google query: https://www.google.com/search?q=B00L...www.amazon.com
Plugin Amazon.com failed
Traceback (most recent call last):
File "calibre\ebooks\metadata\sources\identify.py", line 47, in run
File "calibre\ebooks\metadata\sources\amazon.py", line 1433, in identify
File "calibre\ebooks\metadata\sources\amazon.py", line 1366, in search_search_engine
File "calibre\ebooks\metadata\sources\search_engines.py ", line 234, in google_search
File "calibre\ebooks\metadata\sources\search_engines.py ", line 68, in query
File "mechanize\_mechanize.py", line 241, in open_novisit
File "mechanize\_mechanize.py", line 313, in _mech_open
mechanize._response.get_seek_wrapper_class..httper ror_seek_wrapper: HTTP Error 429: Too Many Requests

************************************************** ******************************
The identify phase took 1.02 seconds
The longest time (1.010996) was taken by: Amazon.com
Merging results from different sources
We have 1 merged results, merging took: 0.00 seconds

================================================== ===================================

[davidfor]

When I add the Steve Berry book with the identifiers in the log, and enabled just the Google and Amazon source enabled, I get:



Which is what I expect.
When I go to the metadata download options and unselect all the fields, I get:



Which sounds like what you are describing. So, check the configuration. You can do this before starting the metadata download by pressing the "Configure download" button. Make sure some fields are selected in both the main section and for the sources you use and try again.

[Rob557]

Originally Posted by davidfor

... just the Google and Amazon source enabled ... check the configuration

Thanks Dave. With the help of your comments I've now realized a couple of things:
1. Goodreads metadata plug-in missing. Although the "download metadata" feature is accessed by a right-click on ePubs and so seems to be an integral feature of Calibre, it also uses second-party plug-ins to access alternative data sources. Under Calibre V4.23 I had a Goodreads metadata plug-in source which was missing under Calibre V5.x so I have now installed the Sept 2020 version of that plug-in.
2. Two levels of metadata configuration. I was confused when you were telling me to check the metadata download configuration because I always specify which data I want downloaded (e.g. tags and/or comments), but I now remember that for certain metadata sources (e.g. amazon.com or goodreads) there is also a source-specific configuration option that applies to some sources. In my prior reliance on goodreads, I had configured the alternative amazon.com source such that although it accessed metadata (as shown in the log) I was not selecting that data to incorporate into my Calibre data base and was only using it as an alternative source for covers.

Problem solved.
Thanks !
Rob

[BetterRed]

AFAIK: Goodreads has always been an optional Metadata source plugin that must be installed.

BR

[Rob557]

Originally Posted by BetterRed

AFAIK: Goodreads has always been an optional Metadata source plugin that must be installed.

Agreed. That would mean I would have previously had the plugin installed and did not realize I had lost it when I moved to V5.5 (it's been quite awhile, but now and again I do a swatch of metadata downloads and goodreads is generally my preferred source). Maybe I'm getting something else mixed up in the sequence of events, but right now all seems to be working okay. Thanks.