Downloading covers (bug?)

[Clem2605]

Hello,

I've just noticed that when I try to download covers for my books in calibre, it no longer displays several covers from various sources (in my case: Google, Google Image and Amazon) but only one cover from Amazon.

I first thought that it was an internet/downloading problem, as it happened before, but usually, it still displays some covers, and not always from Amazon, and it resolve itself when I search again or at most a few hours later/when my connexion gets better.
But I've got no connection problems right now and it's been 48 hours, and I've still been unable to see more than one cover from Amazon every time I do a search.

I could be that there is only an Amazon cover to download on the net for the books I'm looking for, but I was searching for Throne of Glass by Sarah J. Maas, one of the most well-known authors right now, so I doubt it (plus I tried for my Harry Potter books, just to be sure: same results).

Something I noticed is that it doesn't wait 60s after the first cover is found to stop the search, like I set it up, it stop the search really quickly.
I've tried to uncheck Amazon as a source, but it only stop searching really quickly.

From the few tests I've done, I think that it's either a case of the search shutting down too quickly so only Amazon has the time to download, or the other sources are broken.

Here is the log of one of my tries: (My checked sources for this one are Amazon, Google, Google Image and Open Library)

Spoiler:

Starting cover download for: Throne of Glass
Query: Throne of Glass ['Sarah J. Maas'] {}

****************************** Google Images Covers ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:86.0) Gecko/20100101 Firefox/86.0')]
Failed to download valid cover
Took 0.30884408950805664 seconds
Search URL: https://www.google.com/search?as_st=...,iar:t,ift:jpg
No images found for, title: 'Throne of Glass' and authors: ['Sarah J. Maas']

************************************************** ******************************

****************************** Amazon.com Covers ******************************
Request extra headers: [('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8'), ('Upgrade-insecure-requests', '1'), ('Referer', 'https://www.bing.com/search?q=Throne+of+Glass+Sarah+Maas+site%3Awww.ama zon.com'), ('User-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36')]
Failed to download valid cover
Took 0.3988208770751953 seconds
No cached cover found, running identify
User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.119 Safari/537.36
Server: auto
Making google query: https://www.google.com/search?q=Thro...www.amazon.com
Failed to find any results on results page, with title: Avant d'accéder à la recherche*Google
No search engine results for terms: Throne of Glass Sarah Maas
Trying the bing search engine instead
Making bing query: https://www.bing.com/search?q=Throne...www.amazon.com
Failed to find any results on results page, with title: Throne of Glass Sarah Maas site:www.amazon.com - Recherche
No search engine results for terms: Throne of Glass Sarah Maas
No matches found with query: ['Throne', 'of', 'Glass', 'Sarah', 'Maas']
No cover found

************************************************** ******************************

****************************** Open Library Covers ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko')]
Failed to download valid cover
Took 0.0 seconds

************************************************** ******************************

Any help would be welcome!

[kovidgoyal]

Both google images and amazon are returning covers for me. See attached. If waiting for a day or so isnt fixing the issue for you, try rebooting in safe mode, just in case whatever is interfering is running on your computer.

[theducks]

Could this be a Locale issue?
I noticed the log was not all in English.

[kovidgoyal]

Originally Posted by theducks

Could this be a Locale issue?
I noticed the log was not all in English.

Not for something that started failing recently, unless the user changed his locale/location.

[Comfy.n]

Originally Posted by Clem2605

Making google query: https://www.google.com/search?q=Thro...www.amazon.com
Failed to find any results on results page, with title: Avant d'accéder à la recherche*Google

That sounds like Google complaining "How dare you do some bottish search. I can't show you any ads, this way. Eat these cookies! Solve the captcha! You must consent!"

Perhaps something related to EU's GDPR: (not exactly new)

https://www.twipla.com/en/blog/googl...ure-gdpr-fines

[kovidgoyal]

Yeah I think that french translates as "Before accessing the search" so it must be a gdpr thing or a captcha thing. Sadly I dont have a machine in france to test with, but most likely it just needs some gdpr consent cookie to be set. If someone can figure that out and tell me what it is, I will add it. Just do a google search in your browser in france and use the dev tools to inspect the request for all cookies. Assuming its a "global" cookie not a per request one it should be easy.

[Clem2605]

Thanks for the answers!

I can confirm that 'Avant d'accéder à la recherche' means 'Before accessing the search' (I didn't notice that there was french in the log, I would have translated otherwise), and that I'm in France.
I didn't even know that something changed in Europe recently due to GDPR ^^'.
I don't know if the change is that recent: while I used to utilize this feature a lot, I didn't get any new books in the last year, and thus didn't utilize it as much as I used to. I honestly can't remember the last time I tried to use it and it worked (especially since at first I thought that it was just a temporary service problem, I might have encountered it and just shrugged it off the first few times).

So since my location seemed to be the problem, I tried with my VPN set in the US and, magic, it worked!! So, definitely, you found the problem, it's the location.

Honestly, I'm really not good at figuring out web problems, so I don't really know how to

'use the dev tools to inspect the request for all cookies'.

I'll try searching this week-end, to see if I can find what you're looking for.
In the meantime, at least, we have a temporary solution to the problem.

[kovidgoyal]

Open the dev tools in your browser (usual ctrl+shift+i but varies by browser). Then go to the network tab, then paste the searhc url and press enter. The network tab will record all requests. Look at the one that returns the search html, and see if you can spot the cookies set in the request header.