Aeon.co - Recipe download failed - CERTIFICATE_VERIFY_FAILED

[emanu]

Hi,
I'm trying to download feeds from https://aeon.co/ using Calibre. I've creaded recipe with the following link:
https://aeon.co/feed
(or https://aeon.co/feed.rss)

Unfortunately, when downloading such recipe resulting mobi file is empty and only contains a note:

Failed feed: Aeon Magazine
<urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>

I remember using that link in the past and it worked without any problems. But some time ago it stopped. I don't know what the reason is, upgrade of Caliber or changes in Aeon website configuration.

I would be very grateful for any help with regard to this issue.

Thank you !

[kovidgoyal]

That error means that the feed is using an invalid https certificate. Use the pure http version of the URL if available or contact them and get them to fix their certificate

[emanu]

Hi Kovidgoyal,
Thank you very much for your explanation. I've just tried http version of their address and it doesn't work. Let me contact Aeon admin for his ideas on fixing the issue.
Is there any way around that problem that I could use in the interim? That would probably mean digging deep in the recipe source code so I'm not sure it's worthwile.

Thank you and I will let you know if Aeon provides any relevant comments.

emanu

[emanu]

Let me add one more comment. When I put RSS link directly in the browser I can see the content being displayed - the page loads, I can see the list of headers and some short descriptions. But when I use the very same link to create Calibre recipe then I get the SSL: CERTIFICATE_VERIFY_FAILED error.

Does it still mean the problem is with the website certificate rather than the way Calibre processes the recipe/link?

Thanks!

[emanu]

The same Certificate problem exist for other website that I use Calibre do download the contnet, e.g.:

Intelligent Life - https://www.1843magazine.com/rss/content
TheBrowser - https://thebrowser.com/feed

Does it mean they all have SSL certificates configured wrongly?

Please advise.
Thanks!

[JSWolf]

Make sure your date/time on your computer is correct. The last time I got such an error was because the date/time was incorrect. Once the date/time was corrected, the certificate was accepted.

[eschwartz]

The HTTPS version of that website downloads fine with wget/curl

So perhaps your computer is simply missing the appropriate certificate. Firefox for one ships with its own internal certificates, so it wouldn't report such an error.

[emanu]

Hi,
Thank you for all suggestions.
Just to add some details - I'm on Win 7 Professional on 64bit computer, patched and updated. I'm using the latest version of FireFox. I have also updated version of Calibre.

If the problem is with certificate on my local computer then how do I update it?

In the meantime I received very comprehensive reply from Aeon.co web developer, who doesn't see the problem on their end:

" I can tell you that the Aeon SSL certificate is working without issues. You can verify it yourself here:
https://www.ssllabs.com/ssltest/anal...hideResults=on

There are a few other possibilities:

* You may be using an old operating system, and with a recent update they may have changed the built-in certificates or SSL library.

* Your version of Calibre may be outdated.

* Calibre may have an outdated SSL library included and may try to connect via SSL v3, like some older applications still do. However — in line with today's best practise — we only support TLS for HTTPS connections.

Unfortunately we no longer support connections via normal HTTP.
"

In relation to this reply I confirm:
- my OS is not old (win 7) and it's updated
- Caliber is also in its latest release
- Calibre library (?) - can't comment on that, need someone from Calibre dev team


The strange thing is that this Rss feed via Calibre recipe used to work. I think it actually might have something to do with Calibre updates which I try to apply regularly. But maybe I should use the approach - "if it ain't broke don't fix it".

Does anyone has any idea on what could be the source of this issue?

Thank you !

[kovidgoyal]

calibre most definitely does not have an outdated ssl library. It uses the SSL library that comes bundled with python 2.7.11. So that leaves your operating system missing the certificates. Rememebr to applya ll optional updates in windows as well -- often certificate updates are optional.

[kovidgoyal]

And here's the output from dumping the certificates used by that site:

openssl s_client -showcerts -connect aeon.co:443 < /dev/null                                                                                                                                                            **1*
CONNECTED(00000005)
depth=4 C = US, O = "Starfield Technologies, Inc.", OU = Starfield Class 2 Certification Authority
verify return:1
depth=3 C = US, ST = Arizona, L = Scottsdale, O = "Starfield Technologies, Inc.", CN = Starfield Services Root Certificate Authority - G2
verify return:1
depth=2 C = US, O = Amazon, CN = Amazon Root CA 1
verify return:1
depth=1 C = US, O = Amazon, OU = Server CA 1B, CN = Amazon
verify return:1
depth=0 CN = aeon.co
verify return:1
---
Certificate chain
 0 s:/CN=aeon.co
   i:/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
-----BEGIN CERTIFICATE-----
MIIETTCCAzWgAwIBAgIQDOuaVhCwTPElm4zRUc3/KTANBgkqhkiG9w0BAQsFADBG
MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRUwEwYDVQQLEwxTZXJ2ZXIg
Q0EgMUIxDzANBgNVBAMTBkFtYXpvbjAeFw0xNjA1MTgwMDAwMDBaFw0xNzA2MTgx
MjAwMDBaMBIxEDAOBgNVBAMTB2Flb24uY28wggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQCn3eqssawWmjwqfoENDXCBAPP/RfXAuEsldKlRPKGKAlGxJaFe
anpFLtHK7Mxtbt75ckhr2KBhshspiZra614xS/Rl9+X0fJmqSYvhMb4de8ugHE+J
7aQVGpC7pBJ3AaC4fj96ZQYhaBu9RDIo44A3EGc8CzKgjxqjoqGcBt3pXKFBe2ss
zjVKUkcyw0ybHHa6Zwmp19HrNLzablcD2wz7S9mxgCD3MhiDtLg1GSKhAWEwpq7G
CXeuo3tKU3YXxpv3y3mdlbh/9/WtjtWp+vy2iy21MYBG0FZPTJ3vQBOodj/EYmVw
Co4pawl2pFqe7FdRQN/J7i3Jvd8cEGCUdRdjAgMBAAGjggFpMIIBZTAfBgNVHSME
GDAWgBRZpGYGUqB7lZI8o5QHJ5Z0W/k90DAdBgNVHQ4EFgQURYoV084IBcHkA4lH
VyVTxKbpm4wwHQYDVR0RBBYwFIIHYWVvbi5jb4IJKi5hZW9uLmNvMA4GA1UdDwEB
/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwOwYDVR0fBDQw
MjAwoC6gLIYqaHR0cDovL2NybC5zY2ExYi5hbWF6b250cnVzdC5jb20vc2NhMWIu
Y3JsMBMGA1UdIAQMMAowCAYGZ4EMAQIBMHUGCCsGAQUFBwEBBGkwZzAtBggrBgEF
BQcwAYYhaHR0cDovL29jc3Auc2NhMWIuYW1hem9udHJ1c3QuY29tMDYGCCsGAQUF
BzAChipodHRwOi8vY3J0LnNjYTFiLmFtYXpvbnRydXN0LmNvbS9zY2ExYi5jcnQw
DAYDVR0TAQH/BAIwADANBgkqhkiG9w0BAQsFAAOCAQEAaWfieowk/dQB/M70l0jF
OzOwazf4mw6YcfV2msWcLVnPlcPC4gRpx+qDu/9u5SjqiEUte82poslEVX7KJJ0x
Py1yVS5fy3v+Q6Wnl4tPn50Rap7cOg8C47sSyGETruz7KjNnIgtLzquFvgeJFQHd
x+zmXOcoLvLkoF+P/xOQ5CdBLQfgBonKtCmRBkCXm7r3+vxX3yJq2OOrs4odThg9
EBIDTypWxHOvR6Siz1ykWZa/AuUuOdHeye/+AL7RrZXmG++oxzhHtZsPA8tRxnGE
tiZqroAmpieSOQl1fWTU25gnouTQDw5Sf9M32XJ+PHu9FbgDWdnCqyFn1vAURabD
6g==
-----END CERTIFICATE-----
 1 s:/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
   i:/C=US/O=Amazon/CN=Amazon Root CA 1
-----BEGIN CERTIFICATE-----
MIIESTCCAzGgAwIBAgITBn+UV4WH6Kx33rJTMlu8mYtWDTANBgkqhkiG9w0BAQsF
ADA5MQswCQYDVQQGEwJVUzEPMA0GA1UEChMGQW1hem9uMRkwFwYDVQQDExBBbWF6
b24gUm9vdCBDQSAxMB4XDTE1MTAyMjAwMDAwMFoXDTI1MTAxOTAwMDAwMFowRjEL
MAkGA1UEBhMCVVMxDzANBgNVBAoTBkFtYXpvbjEVMBMGA1UECxMMU2VydmVyIENB
IDFCMQ8wDQYDVQQDEwZBbWF6b24wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQDCThZn3c68asg3Wuw6MLAd5tES6BIoSMzoKcG5blPVo+sDORrMd4f2AbnZ
cMzPa43j4wNxhplty6aUKk4T1qe9BOwKFjwK6zmxxLVYo7bHViXsPlJ6qOMpFge5
blDP+18x+B26A0piiQOuPkfyDyeR4xQghfj66Yo19V+emU3nazfvpFA+ROz6WoVm
B5x+F2pV8xeKNR7u6azDdU5YVX1TawprmxRC1+WsAYmz6qP+z8ArDITC2FMVy2fw
0IjKOtEXc/VfmtTFch5+AfGYMGMqqvJ6LcXiAhqG5TI+Dr0RtM88k+8XUBCeQ8IG
KuANaL7TiItKZYxK1MMuTJtV9IblAgMBAAGjggE7MIIBNzASBgNVHRMBAf8ECDAG
AQH/AgEAMA4GA1UdDwEB/wQEAwIBhjAdBgNVHQ4EFgQUWaRmBlKge5WSPKOUByeW
dFv5PdAwHwYDVR0jBBgwFoAUhBjMhTTsvAyUlC4IWZzHshBOCggwewYIKwYBBQUH
AQEEbzBtMC8GCCsGAQUFBzABhiNodHRwOi8vb2NzcC5yb290Y2ExLmFtYXpvbnRy
dXN0LmNvbTA6BggrBgEFBQcwAoYuaHR0cDovL2NydC5yb290Y2ExLmFtYXpvbnRy
dXN0LmNvbS9yb290Y2ExLmNlcjA/BgNVHR8EODA2MDSgMqAwhi5odHRwOi8vY3Js
LnJvb3RjYTEuYW1hem9udHJ1c3QuY29tL3Jvb3RjYTEuY3JsMBMGA1UdIAQMMAow
CAYGZ4EMAQIBMA0GCSqGSIb3DQEBCwUAA4IBAQCFkr41u3nPo4FCHOTjY3NTOVI1
59Gt/a6ZiqyJEi+752+a1U5y6iAwYfmXss2lJwJFqMp2PphKg5625kXg8kP2CN5t
6G7bMQcT8C8xDZNtYTd7WPD8UZiRKAJPBXa30/AbwuZe0GaFEQ8ugcYQgSn+IGBI
8/LwhBNTZTUVEWuCUUBVV18YtbAiPq3yXqMB48Oz+ctBWuZSkbvkNodPLamkB2g1
upRyzQ7qDn1X8nn8N8V7YJ6y68AtkHcNSRAnpTitxBKjtKPISLMVCx7i4hncxHZS
yLyKQXhw2W2Xs0qLeC1etA+jTGDK4UfLeC0SF7FSi8o5LL21L8IzApar2pR/
-----END CERTIFICATE-----
 2 s:/C=US/O=Amazon/CN=Amazon Root CA 1
   i:/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./CN=Starfield Services Root Certificate Authority - G2
-----BEGIN CERTIFICATE-----
MIIEkjCCA3qgAwIBAgITBn+USionzfP6wq4rAfkI7rnExjANBgkqhkiG9w0BAQsF
ADCBmDELMAkGA1UEBhMCVVMxEDAOBgNVBAgTB0FyaXpvbmExEzARBgNVBAcTClNj
b3R0c2RhbGUxJTAjBgNVBAoTHFN0YXJmaWVsZCBUZWNobm9sb2dpZXMsIEluYy4x
OzA5BgNVBAMTMlN0YXJmaWVsZCBTZXJ2aWNlcyBSb290IENlcnRpZmljYXRlIEF1
dGhvcml0eSAtIEcyMB4XDTE1MDUyNTEyMDAwMFoXDTM3MTIzMTAxMDAwMFowOTEL
MAkGA1UEBhMCVVMxDzANBgNVBAoTBkFtYXpvbjEZMBcGA1UEAxMQQW1hem9uIFJv
b3QgQ0EgMTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBALJ4gHHKeNXj
ca9HgFB0fW7Y14h29Jlo91ghYPl0hAEvrAIthtOgQ3pOsqTQNroBvo3bSMgHFzZM
9O6II8c+6zf1tRn4SWiw3te5djgdYZ6k/oI2peVKVuRF4fn9tBb6dNqcmzU5L/qw
IFAGbHrQgLKm+a/sRxmPUDgH3KKHOVj4utWp+UhnMJbulHheb4mjUcAwhmahRWa6
VOujw5H5SNz/0egwLX0tdHA114gk957EWW67c4cX8jJGKLhD+rcdqsq08p8kDi1L
93FcXmn/6pUCyziKrlA4b9v7LWIbxcceVOF34GfID5yHI9Y/QCB/IIDEgEw+OyQm
jgSubJrIqg0CAwEAAaOCATEwggEtMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/
BAQDAgGGMB0GA1UdDgQWBBSEGMyFNOy8DJSULghZnMeyEE4KCDAfBgNVHSMEGDAW
gBScXwDfqgHXMCs4iKK4bUqc8hGRgzB4BggrBgEFBQcBAQRsMGowLgYIKwYBBQUH
MAGGImh0dHA6Ly9vY3NwLnJvb3RnMi5hbWF6b250cnVzdC5jb20wOAYIKwYBBQUH
MAKGLGh0dHA6Ly9jcnQucm9vdGcyLmFtYXpvbnRydXN0LmNvbS9yb290ZzIuY2Vy
MD0GA1UdHwQ2MDQwMqAwoC6GLGh0dHA6Ly9jcmwucm9vdGcyLmFtYXpvbnRydXN0
LmNvbS9yb290ZzIuY3JsMBEGA1UdIAQKMAgwBgYEVR0gADANBgkqhkiG9w0BAQsF
AAOCAQEAYjdCXLwQtT6LLOkMm2xF4gcAevnFWAu5CIw+7bMlPLVvUOTNNWqnkzSW
MiGpSESrnO09tKpzbeR/FoCJbM8oAxiDR3mjEH4wW6w7sGDgd9QIpuEdfF7Au/ma
eyKdpwAJfqxGF4PcnCZXmTA5YpaP7dreqsXMGz7KQ2hsVxa81Q4gLv7/wmpdLqBK
bRRYh5TmOTFffHPLkIhqhBGWJ6bt2YFGpn6jcgAKUj6DiAdjd4lpFw85hdKrCEVN
0FE6/V1dN2RMfjCyVSRCnTawXZwXgWHxyvkQAiSr6w10kY17RSlQOYiypok1JR4U
akcjMS9cmvqtmg5iUaQqqcT5NJ0hGA==
-----END CERTIFICATE-----
 3 s:/C=US/ST=Arizona/L=Scottsdale/O=Starfield Technologies, Inc./CN=Starfield Services Root Certificate Authority - G2
   i:/C=US/O=Starfield Technologies, Inc./OU=Starfield Class 2 Certification Authority
-----BEGIN CERTIFICATE-----
MIIEdTCCA12gAwIBAgIJAKcOSkw0grd/MA0GCSqGSIb3DQEBCwUAMGgxCzAJBgNV
BAYTAlVTMSUwIwYDVQQKExxTdGFyZmllbGQgVGVjaG5vbG9naWVzLCBJbmMuMTIw
MAYDVQQLEylTdGFyZmllbGQgQ2xhc3MgMiBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0
eTAeFw0wOTA5MDIwMDAwMDBaFw0zNDA2MjgxNzM5MTZaMIGYMQswCQYDVQQGEwJV
UzEQMA4GA1UECBMHQXJpem9uYTETMBEGA1UEBxMKU2NvdHRzZGFsZTElMCMGA1UE
ChMcU3RhcmZpZWxkIFRlY2hub2xvZ2llcywgSW5jLjE7MDkGA1UEAxMyU3RhcmZp
ZWxkIFNlcnZpY2VzIFJvb3QgQ2VydGlmaWNhdGUgQXV0aG9yaXR5IC0gRzIwggEi
MA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDVDDrEKvlO4vW+GZdfjohTsR8/
y8+fIBNtKTrID30892t2OGPZNmCom15cAICyL1l/9of5JUOG52kbUpqQ4XHj2C0N
Tm/2yEnZtvMaVq4rtnQU68/7JuMauh2WLmo7WJSJR1b/JaCTcFOD2oR0FMNnngRo
Ot+OQFodSk7PQ5E751bWAHDLUu57fa4657wx+UX2wmDPE1kCK4DMNEffud6QZW0C
zyyRpqbn3oUYSXxmTqM6bam17jQuug0DuDPfR+uxa40l2ZvOgdFFRjKWcIfeAg5J
Q4W2bHO7ZOphQazJ1FTfhy/HIrImzJ9ZVGif/L4qL8RVHHVAYBeFAlU5i38FAgMB
AAGjgfAwge0wDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAYYwHQYDVR0O
BBYEFJxfAN+qAdcwKziIorhtSpzyEZGDMB8GA1UdIwQYMBaAFL9ft9HO3R+G9FtV
rNzXEMIOqYjnME8GCCsGAQUFBwEBBEMwQTAcBggrBgEFBQcwAYYQaHR0cDovL28u
c3MyLnVzLzAhBggrBgEFBQcwAoYVaHR0cDovL3guc3MyLnVzL3guY2VyMCYGA1Ud
HwQfMB0wG6AZoBeGFWh0dHA6Ly9zLnNzMi51cy9yLmNybDARBgNVHSAECjAIMAYG
BFUdIAAwDQYJKoZIhvcNAQELBQADggEBACMd44pXyn3pF3lM8R5V/cxTbj5HD9/G
VfKyBDbtgB9TxF00KGu+x1X8Z+rLP3+QsjPNG1gQggL4+C/1E2DUBc7xgQjB3ad1
l08YuW3e95ORCLp+QCztweq7dp4zBncdDQh/U90bZKuCJ/Fp1U1ervShw3WnWEQt
8jxwmKy6abaVd38PMV4s/KCHOkdp8Hlf9BRUpJVeEXgSYCfOn8J3/yNTd126/+pZ
59vPr5KW7ySaNRB6nJHGDn2Z9j8Z3/VyVOEVqQdZe4O/Ui5GjLIAZHYcSNPYeehu
VsyuLAOQ1xk4meTKCRlb/weWsKh/NEnfVqn3sF/tM+2MR7cwA130A4w=
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=aeon.co
issuer=/C=US/O=Amazon/OU=Server CA 1B/CN=Amazon
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 5188 bytes and written 433 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: E6E6AEEEC288A6367BE3492AE8D130E3BCA52EB8BB42C89AC1D1150F2536841C
    Session-ID-ctx: 
    Master-Key: BB6E7D2C5BDA3A1FFCA6E0F4228D4930EDE7429ACB1A16FB65433D101B00BD8F5A8D699058D60CA7C71DF48E978F1D28
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 7a 18 52 6a f2 7f 07 44-6d fb 16 47 ea e6 d8 2e   z.Rj...Dm..G....
    0010 - 90 5b 40 47 f7 be 3b 4c-22 ff 4f 29 3a f6 f4 63   .[@G..;L".O):..c
    0020 - e1 73 9d bf 9a b8 57 23-8b 3f fb 5d d8 d6 b9 eb   .s....W#.?.]....
    0030 - 52 73 a7 59 09 2c d1 2b-88 f7 de 8f f5 80 dc 72   Rs.Y.,.+.......r
    0040 - 6c 6f c3 76 df d4 d2 c7-89 0a 3d bc 6e 83 a7 11   lo.v......=.n...
    0050 - 64 3d 0e a2 35 dd bd 13-b7 ee de 07 4f 9a 89 ad   d=..5.......O...
    0060 - 92 c2 62 fc 65 db 57 da-6d b8 8e 53 8a 02 12 1a   ..b.e.W.m..S....
    0070 - dc 6e 73 1e cd c5 04 cd-92 64 c4 eb b2 45 b4 d2   .ns......d...E..
    0080 - f7 84 fa 5d 20 91 81 a3-e5 99 53 0c 8e d2 39 33   ...] .....S...93
    0090 - ed 62 20 f7 9e 44 d1 92-f3 66 5f ea 16 bd 15 46   .b ..D...f_....F

    Start Time: 1468593649
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
DONE

You have to check that all the certificates listed in that chain (after the first one, which is the site certificate itself) are installed on your computer.

[kovidgoyal]

On my windows 10 machine, I had to visit the https://aeon.co website in Microsoft Edge, which automatically installed the certificates, and then it worked in calibre as well.

You'll probably have to do it manually in windows 7

[emanu]

Hi Kovidgoyal,
Thank you for your help. With your hints I finally managed to get it work.
Strangly enought when I opened sites links in IExplorer and then tried to download recipes in Calibre it worked ! I did the same trick with couple of other sites with which certification verification error was triggered.

Perfect! I can once again enjoy rss content downloaded on my kindle.

Thanks a lot for your help and spot-on advices - much appreciated

Kind regards.

[eschwartz]

Why on earth would Windows make the certificate store controlled by the browser?

Happy to hear you got it fixed -- and thanks for the status update.

[kovidgoyal]

Originally Posted by eschwartz

Why on earth would Windows make the certificate store controlled by the browser?

Dont ask me -- I've been working on setting up an automatic build system on windows the last few days and the number of strange windows behaviors I have had to deal with is not funny.

[rant]
Their file system performance is an order of magnitude worse than either linux or os x inside a VirtualBox VM.

They just released a compiler update that crashes on perfectly valid C++ files that the previous version of the compiler compiled with no problem.

The windows update client used 1.5 GB of RAM and took an hour and half of continuous processor 100% usage just to decide what updates are available!

Their make equivalent does not support parallel building.

They keep showing you ads for windows 10

EDIT: Oh yeah, and you need 11GB! of diskspace to use their compiler (though this particular pathology is common to OS X as well)
[/rant]