SSL issues with metadata download

[Ybother]

I've looked through the forum and have found other similar questions about this topic. But seeing as I'm in an awkward place here, I'm going to re-ask with the details pertinent to my specific problem.

I'm running Calibre portable 6.24
When I try to download metadata, the query fails. Logs show the following:

Running identify query with parameters:
{'title': 'The Adventures of Sherlock Holmes', 'authors': ['Sir Arthur Conan Doyle'], 'identifiers': {'uri': 'http://www.gutenberg.org/ebooks/1661'}, 'timeout': 30}
Using plugins: Google (1, 1, 1), Amazon.com (1, 3, 5), Edelweiss (2, 0, 1)
The log from individual plugins is below

****************************** Google (1, 1, 1) ******************************
Found 0 results
Downloading from Google took 1.3675224781036377
Making query: https://books.google.com/books/feeds...ewability=none
Failed to make identify query: 'https://books.google.com/books/feeds/volumes?q=intitle%3AAdventures+intitle%3Aof+intitl e%3ASherlock+intitle%3AHolmes%2Binauthor%3ASir+ina uthor%3AArthur+inauthor%3AConan+inauthor%3ADoyle&m ax-results=20&start-index=1&min-viewability=none'
Traceback (most recent call last):
File "mechanize\_urllib2_fork.py", line 1236, in do_open
File "http\client.py", line 1282, in request
File "http\client.py", line 1328, in _send_request
File "http\client.py", line 1277, in endheaders
File "http\client.py", line 1037, in _send_output
File "http\client.py", line 975, in send
File "http\client.py", line 1454, in connect
File "ssl.py", line 512, in wrap_socket
File "ssl.py", line 1070, in _create
File "ssl.py", line 1341, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "calibre\ebooks\metadata\sources\google.py", line 515, in make_query
File "mechanize\_mechanize.py", line 241, in open_novisit
File "mechanize\_mechanize.py", line 287, in _mech_open
File "mechanize\_opener.py", line 193, in open
File "mechanize\_urllib2_fork.py", line 425, in _open
File "mechanize\_urllib2_fork.py", line 414, in _call_chain
File "calibre\utils\browser.py", line 28, in https_open
File "mechanize\_urllib2_fork.py", line 1240, in do_open
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: EE certificate key too weak (_ssl.c:997)>

************************************************** ******************************

(The Amazon and Edelweiss queries fail with the same error, so for brevity I didn't post them)

I'm working from behind a firewall/content filter that parses SSL, similar to what some AV software does. I have no way to turn this off or add an exception, as this is not managed by me.

The firewall re-signs the traffic using its own CA certificate, which I have.
Is there a way to set Calibre to use that CA to accept the resigned SSL queries?

[DNSB]

There might be a way to get calibre to accept unsafe certificates but it would be better to have the certificate using a longer key. Basically, the certificate is using less than 128 bits which is the absolute minimum with OpenSSL security level 2. Ask your IT people to ensure that you are meeting current minimum security standards. Corporately, we use Palo Alto and 4096 bit RSA keys for our SSL decryption internal certificates. OTOH, we don't bother to try to use SSL decryption on Google who are using certificate pinning to make SSL decryption less than useful. There are several other techniques used as well. Our exception list ran to over 4 pages at one point. Sadly, our SSL decryption is pretty much the definition of a MITM attack.

[Ybother]

Thanks David, but here's the issue:
The CA certificate is NOT weak, It's 2048 bit RSA. I can attach a copy of it here if it'll help anyone.

You're right in that the filter essentially does an MTM attack. But since I actually want the service it does, I have no problem with it.

I saw somewhere else on the forum that it's possible to get Calibre to use its own CA store, but I couldn't figure out how to do it and if it works also on the portable version.

[DNSB]

Originally Posted by Ybother

Thanks David, but here's the issue:
The CA certificate is NOT weak, It's 2048 bit RSA. I can attach a copy of it here if it'll help anyone.

You're right in that the filter essentially does an MTM attack. But since I actually want the service it does, I have no problem with it.

I saw somewhere else on the forum that it's possible to get Calibre to use its own CA store, but I couldn't figure out how to do it and if it works also on the portable version.

The error you are getting says nothing about the certificate chain. What it is complaining is that the key size is too short (a 2048 bit RSA key works out to a ~116 bit symmetric key which gets rounded down to 112 bits for easy comparison). What do you see when you look at the certificate? I've attached a couple of images of what I see in Firefox connected to MobileRead and StackOverflow. You will notice the MR image mentions 256 bit keys/TLS 1.3 while StackOverflow uses 128 bit keys and TLS 1.2. This does mean that it takes approximately 32ms to handshake with StackOverflow and 47ms to handshake with MobileRead. The horrors of that one time 15ms of extra time!

It's also possible that you are missing an intermediate certificate but since your corporate CA is being used, you should have been sent the certificate chain.

<deleted a mass of over information> since I think I just bored everybody within 100 metres into a coma.

[kovidgoyal]

calibre uses python which uses OpenSSL. OpenSSL supports env vars to specify certificates. Whether those work on windows I have no idea. IIRC by default SSL uses the windows OS certificate store.

[Ybother]

Thanks for the responses.

I'm comparing my certificate to others, and it looks no different than, say Symantec's root certificate. This certificate is also installed in my Windows certificate store, so Chrome and such have no issues with the resigned certificates. I find that only third-party software that expects a specific certificate (like Google Drive) has issues with this setup. Firefox uses its own store and I have the certificate installed there as well with no issues.

I'm attaching my firewall's CA certificate, as well as the certificate I extracted from Mobileread, in case that gives anyone ideas on how to proceed. (They've both been RARed so they can be attached here...)
_.mobileread.com.rar

Symen (72295).rar

[kovidgoyal]

THis has nothing to do with the CA certificate. The error means the ceritficate for the site your are visiting is too weak, which means your MITM is returning fake certificates that are too weak.