Unable to download metadata/covers, SSL error

[soratoyuki]

Hello! I just updated to Calibre 9.8 and I'm no longer able to download covers and metadata. The upgrade may be a coicidence though. I've tried restarting Calibre and downloading metadata/covers seperately, but no luck. I assume the relevant part of the logs is this message: ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081)

Edited to add: Just realized this might be the first time using Calibre since updating to Fedora 44. Perhaps that's the issue?

Spoiler:

The error message:
calibre, version 9.8.0
ERROR: Download failed: Failed to download metadata or cover for the selected book.

Snow Crash (Failed metadata) (Failed cover)

The job details:

#################### Log for Snow Crash ####################
Running identify query with parameters:
{'title': 'Snow Crash', 'authors': ['Neal Stephenson'], 'identifiers': {'asin': 'B002RI9KAE', 'mobi-asin': 'B002RI9KAE', 'google': 'B0xbAAAAMAAJ', 'isbn': '9780553088533'}, 'timeout': 30}
Using plugins: Google (1, 1, 3), Amazon.com (1, 3, 18)
The log from individual plugins is below

****************************** Google (1, 1, 3) ******************************
Found 0 results
Downloading from Google took 0.11369466781616211
Failed to get metadata for Google identifier: B0xbAAAAMAAJ
Traceback (most recent call last):
File "mechanize/_urllib2_fork.py", line 1242, in do_open
File "http/client.py", line 1358, in request
File "http/client.py", line 1404, in _send_request
File "http/client.py", line 1353, in endheaders
File "http/client.py", line 1113, in _send_output
File "http/client.py", line 1057, in send
File "http/client.py", line 1499, in connect
File "ssl.py", line 455, in wrap_socket
File "ssl.py", line 1076, in _create
File "ssl.py", line 1372, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre/ebooks/metadata/sources/google.py", line 499, in identify
File "calibre/ebooks/metadata/sources/google.py", line 119, in to_metadata
File "calibre/ebooks/metadata/sources/google.py", line 108, in get_extra_details
File "calibre/ebooks/metadata/sources/google.py", line 53, in get_details
File "mechanize/_mechanize.py", line 241, in open_novisit
File "mechanize/_mechanize.py", line 287, in _mech_open
File "mechanize/_opener.py", line 193, in open
File "mechanize/_urllib2_fork.py", line 431, in _open
File "mechanize/_urllib2_fork.py", line 420, in _call_chain
File "calibre/utils/browser.py", line 28, in https_open
File "mechanize/_urllib2_fork.py", line 1246, in do_open
urllib.error.URLError:

Making query: https://books.google.com/books/feeds...ewability=none
Failed to make identify query: 'https://books.google.com/books/feeds/volumes?q=isbn%3A9780553088533&max-results=20&start-index=1&min-viewability=none'
Traceback (most recent call last):
File "mechanize/_urllib2_fork.py", line 1242, in do_open
File "http/client.py", line 1358, in request
File "http/client.py", line 1404, in _send_request
File "http/client.py", line 1353, in endheaders
File "http/client.py", line 1113, in _send_output
File "http/client.py", line 1057, in send
File "http/client.py", line 1499, in connect
File "ssl.py", line 455, in wrap_socket
File "ssl.py", line 1076, in _create
File "ssl.py", line 1372, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre/ebooks/metadata/sources/google.py", line 518, in make_query
File "mechanize/_mechanize.py", line 241, in open_novisit
File "mechanize/_mechanize.py", line 287, in _mech_open
File "mechanize/_opener.py", line 193, in open
File "mechanize/_urllib2_fork.py", line 431, in _open
File "mechanize/_urllib2_fork.py", line 420, in _call_chain
File "calibre/utils/browser.py", line 28, in https_open
File "mechanize/_urllib2_fork.py", line 1246, in do_open
urllib.error.URLError:

************************************************** ******************************

****************************** Amazon.com (1, 3, 18) ******************************
Found 0 results
Downloading from Amazon.com took 0.051542043685913086
User-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:147.0) Gecko/20100101 Firefox/147.0
Server: auto
Making google query: https://www.google.com/search?q=B002...zon.com&nfpr=1
Plugin Amazon.com failed
Traceback (most recent call last):
File "mechanize/_urllib2_fork.py", line 1242, in do_open
File "http/client.py", line 1358, in request
File "http/client.py", line 1404, in _send_request
File "http/client.py", line 1353, in endheaders
File "http/client.py", line 1113, in _send_output
File "http/client.py", line 1057, in send
File "http/client.py", line 1499, in connect
File "ssl.py", line 455, in wrap_socket
File "ssl.py", line 1076, in _create
File "ssl.py", line 1372, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre/ebooks/metadata/sources/identify.py", line 47, in run
File "calibre/ebooks/metadata/sources/amazon.py", line 1675, in identify
File "calibre/ebooks/metadata/sources/amazon.py", line 1603, in search_search_engine
File "calibre/ebooks/metadata/sources/search_engines.py", line 415, in google_search
File "calibre/ebooks/metadata/sources/search_engines.py", line 100, in query
File "mechanize/_mechanize.py", line 241, in open_novisit
File "mechanize/_mechanize.py", line 287, in _mech_open
File "mechanize/_opener.py", line 193, in open
File "mechanize/_urllib2_fork.py", line 431, in _open
File "mechanize/_urllib2_fork.py", line 420, in _call_chain
File "calibre/utils/browser.py", line 28, in https_open
File "mechanize/_urllib2_fork.py", line 1246, in do_open
urllib.error.URLError:

************************************************** ******************************
The identify phase took 0.20 seconds
The longest time (0.113695) was taken by: Google
Merging results from different sources
We have 0 merged results, merging took: 0.00 seconds
Failed to download metadata for Snow Crash

****************************** Amazon.com Covers ******************************
Request extra headers: [('User-agent', 'Lynx/2.8.6rel.5 libwww-FM/2.14'), ('Accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8'), ('Upgrade-insecure-requests', '1')]
Failed to download valid cover
Took 2.0815200805664062 seconds
No cached cover found, running identify
User-agent: Lynx/2.8.6rel.5 libwww-FM/2.14
Server: auto
Making google query: https://www.google.com/search?q=B002...zon.com&nfpr=1
Failed to download cover from Amazon.com
Traceback (most recent call last):
File "mechanize/_urllib2_fork.py", line 1242, in do_open
File "http/client.py", line 1358, in request
File "http/client.py", line 1404, in _send_request
File "http/client.py", line 1353, in endheaders
File "http/client.py", line 1113, in _send_output
File "http/client.py", line 1057, in send
File "http/client.py", line 1499, in connect
File "ssl.py", line 455, in wrap_socket
File "ssl.py", line 1076, in _create
File "ssl.py", line 1372, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre/ebooks/metadata/sources/covers.py", line 46, in run
File "calibre/ebooks/metadata/sources/amazon.py", line 1762, in download_cover
File "calibre/ebooks/metadata/sources/amazon.py", line 1675, in identify
File "calibre/ebooks/metadata/sources/amazon.py", line 1603, in search_search_engine
File "calibre/ebooks/metadata/sources/search_engines.py", line 415, in google_search
File "calibre/ebooks/metadata/sources/search_engines.py", line 100, in query
File "mechanize/_mechanize.py", line 241, in open_novisit
File "mechanize/_mechanize.py", line 287, in _mech_open
File "mechanize/_opener.py", line 193, in open
File "mechanize/_urllib2_fork.py", line 431, in _open
File "mechanize/_urllib2_fork.py", line 420, in _call_chain
File "calibre/utils/browser.py", line 28, in https_open
File "mechanize/_urllib2_fork.py", line 1246, in do_open
urllib.error.URLError:

************************************************** ******************************

****************************** Open Library Covers ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:148.0) Gecko/20100101 Firefox/148.0')]
Failed to download valid cover
Took 0.18747687339782715 seconds
Failed to download cover for ISBN: 9780553088533
Traceback (most recent call last):
File "mechanize/_urllib2_fork.py", line 1242, in do_open
File "http/client.py", line 1358, in request
File "http/client.py", line 1404, in _send_request
File "http/client.py", line 1353, in endheaders
File "http/client.py", line 1113, in _send_output
File "http/client.py", line 1057, in send
File "http/client.py", line 1499, in connect
File "ssl.py", line 455, in wrap_socket
File "ssl.py", line 1076, in _create
File "ssl.py", line 1372, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1081)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre/ebooks/metadata/sources/openlibrary.py", line 30, in download_cover
File "mechanize/_mechanize.py", line 241, in open_novisit
File "mechanize/_mechanize.py", line 287, in _mech_open
File "mechanize/_opener.py", line 193, in open
File "mechanize/_urllib2_fork.py", line 431, in _open
File "mechanize/_urllib2_fork.py", line 420, in _call_chain
File "calibre/utils/browser.py", line 28, in https_open
File "mechanize/_urllib2_fork.py", line 1246, in do_open
urllib.error.URLError:

************************************************** ******************************

[kovidgoyal]

https://bugs.launchpad.net/calibre/+bug/2151819

[soratoyuki]

Awesome, thank you!