Libération - [SSL: CERTIFICATE_VERIFY_FAILED]

[mpf]

Hi guys,

I've been trying to download news from Libération for quite some time but I always get the same error:

InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:liberation
Trying to get latest version of recipe: liberation
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36
Traceback (most recent call last):
File "mechanize\_urllib2_fork.py", line 1236, in do_open
File "http\client.py", line 1255, in request
File "http\client.py", line 1301, in _send_request
File "http\client.py", line 1250, in endheaders
File "http\client.py", line 1010, in _send_output
File "http\client.py", line 950, in send
File "http\client.py", line 1424, in connect
File "ssl.py", line 500, in wrap_socket
File "ssl.py", line 1040, in _create
File "ssl.py", line 1309, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "runpy.py", line 194, in _run_module_as_main
File "runpy.py", line 87, in _run_code
File "site.py", line 82, in <module>
File "site.py", line 77, in main
File "site.py", line 49, in run_entry_point
File "calibre\utils\ipc\worker.py", line 216, in main
File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
File "calibre\ebooks\conversion\plumber.py", line 1105, in run
File "calibre\customize\conversion.py", line 244, in __call__
File "calibre\ebooks\conversion\plugins\recipe_input.py ", line 137, in convert
File "calibre\web\feeds\news.py", line 931, in __init__
File "<string>", line 45, in get_browser
File "mechanize\_mechanize.py", line 257, in open
File "mechanize\_mechanize.py", line 287, in _mech_open
File "mechanize\_opener.py", line 204, in open
File "mechanize\_urllib2_fork.py", line 535, in http_response
File "mechanize\_opener.py", line 222, in error
File "mechanize\_urllib2_fork.py", line 414, in _call_chain
File "mechanize\_urllib2_fork.py", line 655, in http_error_302
File "mechanize\_mechanize.py", line 257, in open
File "mechanize\_mechanize.py", line 287, in _mech_open
File "mechanize\_opener.py", line 193, in open
File "mechanize\_urllib2_fork.py", line 425, in _open
File "mechanize\_urllib2_fork.py", line 414, in _call_chain
File "calibre\utils\browser.py", line 29, in https_open
File "mechanize\_urllib2_fork.py", line 1240, in do_open
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)>

I'm using Calibre 5.30 64bit on Windows 10. I've tried this on previous versions of Calibre and the same error occurred. I've checked this forum for similar problems and I've found this possible solution: https://www.mobileread.com/forums/sh...postcount=1139

However, I don't have the Intermediate R3 certificate on my system, so the solution is not viable for me. Just to be sure, I've deleted also the DST Root CA X3 and tried again, but the same error occurred.

Thank you for any ideas on how to solve this.

[kovidgoyal]

visit the site using edge or internet explorer

[Jenga]

Hello kovidgoyal,

I've had trouble using Calibre to download either Le Monde or Liberation, and I've seen you give advice in both threads I've found on this forum.

Originally Posted by kovidgoyal

visit the site using edge or internet explorer

What do you mean by that ? Literally visiting the websites of the newspaper? Or getting Calibre to do so, as a reference to this line:
>Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36

If so, I don't know how to do it. Could you help me? Here is a copy of my error message for Le Monde, I'm getting basically the same for Liberation (although a free french newspaper, 20 Minutes, works without problem)

calibre, version 5.30.0 (win32, embedded-python: True)
Erreur lors de la conversion: Échoué: Récupérer des actualités à partir de Le Monde

Récupérer des actualités à partir de Le Monde
Conversion options changed from defaults:
verbose: 2
output_profile: 'tablet'
Resolved conversion options
calibre version: 5.30.0
{'approximate_pages': False,
'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'cde_type_pdoc': False,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'enable_timeout': False,
'expand_css': False,
'extra_css': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x000002C931170BE0>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'no_chapters_in_toc': False,
'no_inline_navbars': False,
'number_of_pages_field': '(auto)',
'output_profile': <calibre.customize.profiles.TabletOutput object at 0x000002C9311835B0>,
'page_breaks_before': None,
'prefer_metadata_cover': False,
'pretty_print': True,
'pubdate': None,
'publisher': None,
'quality_report': False,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'show_kpr_logs': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:le_monde
Trying to get latest version of recipe: le_monde
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36
Traceback (most recent call last):
File "runpy.py", line 194, in _run_module_as_main
File "runpy.py", line 87, in _run_code
File "site.py", line 82, in <module>
File "site.py", line 77, in main
File "site.py", line 49, in run_entry_point
File "calibre\utils\ipc\worker.py", line 216, in main
File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
File "calibre\ebooks\conversion\plumber.py", line 1105, in run
File "calibre\customize\conversion.py", line 244, in __call__
File "calibre\ebooks\conversion\plugins\recipe_input.py ", line 137, in convert
File "calibre\web\feeds\news.py", line 931, in __init__
File "<string>", line 96, in get_browser
File "mechanize\_mechanize.py", line 681, in select_form
mechanize._mechanize.FormNotFoundError: no form matching name 'connection'

[kovidgoyal]

That's a different error and indicates the website has changed, and the recipe needs to be updated.

[Jenga]

Hi,

Thank you for your answer. Did you change something? I just tried again and managed to download the Le Monde feed. The problem is still there for Libération, though.

I get this error, which seems to me like the same as before for Le Monde? I may be mistaken:

calibre, version 5.30.0 (win32, embedded-python: True)
Erreur lors de la conversion: Échoué: Récupérer des actualités à partir de Liberation

Récupérer des actualités à partir de Liberation
Conversion options changed from defaults:
verbose: 2
output_profile: 'tablet'
Resolved conversion options
calibre version: 5.30.0
{'approximate_pages': False,
'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'cde_type_pdoc': False,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'enable_timeout': False,
'expand_css': False,
'extra_css': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x000002DA14C7FC10>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'no_chapters_in_toc': False,
'no_inline_navbars': False,
'number_of_pages_field': '(auto)',
'output_profile': <calibre.customize.profiles.TabletOutput object at 0x000002DA14C90550>,
'page_breaks_before': None,
'prefer_metadata_cover': False,
'pretty_print': True,
'pubdate': None,
'publisher': None,
'quality_report': False,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'show_kpr_logs': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:liberation
Trying to get latest version of recipe: liberation
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36
Traceback (most recent call last):
File "runpy.py", line 194, in _run_module_as_main
File "runpy.py", line 87, in _run_code
File "site.py", line 82, in <module>
File "site.py", line 77, in main
File "site.py", line 49, in run_entry_point
File "calibre\utils\ipc\worker.py", line 216, in main
File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
File "calibre\ebooks\conversion\plumber.py", line 1105, in run
File "calibre\customize\conversion.py", line 244, in __call__
File "calibre\ebooks\conversion\plugins\recipe_input.py ", line 137, in convert
File "calibre\web\feeds\news.py", line 931, in __init__
File "<string>", line 46, in get_browser
File "mechanize\_mechanize.py", line 681, in select_form
mechanize._mechanize.FormNotFoundError: no form matching nr 0

Thanks again for your help

[kovidgoyal]

I didnt, the author of the recipe did: https://github.com/kovidgoyal/calibr...589d046db8686c

[mpf]

Hello guys,

Thank you kovidgoyal for your answer. Just as Jenga, I've followed your advice and now the error has changed:

calibre, version 5.31.1 (win32, embedded-python: True)
Erro ao converter: Falha: Obter notícias de Liberation

Obter notícias de Liberation
Conversion options changed from defaults:
output_profile: 'kindle_pw3'
verbose: 2
Resolved conversion options
calibre version: 5.31.1
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_compress': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x000002B15D620880>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': 'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': True,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.KindlePaperWhite3Outpu t object at 0x000002B15D620E80>,
'page_breaks_before': None,
'personal_doc': '[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:liberation
Trying to get latest version of recipe: liberation
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36
Traceback (most recent call last):
File "runpy.py", line 194, in _run_module_as_main
File "runpy.py", line 87, in _run_code
File "site.py", line 82, in <module>
File "site.py", line 77, in main
File "site.py", line 49, in run_entry_point
File "calibre\utils\ipc\worker.py", line 216, in main
File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
File "calibre\ebooks\conversion\plumber.py", line 1105, in run
File "calibre\customize\conversion.py", line 244, in __call__
File "calibre\ebooks\conversion\plugins\recipe_input.py ", line 137, in convert
File "calibre\web\feeds\news.py", line 931, in __init__
File "<string>", line 46, in get_browser
File "mechanize\_mechanize.py", line 681, in select_form
mechanize._mechanize.FormNotFoundError: no form matching nr 0

It is the same error that Jenga was getting for Le Monde (although weirdly Le Monde was working just fine for me). From what I've understand, the problem won't be solved until the author of the recipe for Libération update it, is that it?

[kovidgoyal]

yes, the recipe needs to be fixed.

[Aimylios]

I don't read Libération myself, but since it didn't look too difficult to fix I've just submitted an update: https://github.com/kovidgoyal/calibre/pull/1519

The login mechanism is still not working properly. As a consequence, Calibre will only download free articles. But at least there should be no more errors about SSL, FormNotFoundError, etc. I'll leave that for someone else to fix.

[Jenga]

Thanks Aimylios, it's downloading! Hopefully the author of the recipe sees this thread and finds a fix for paying articles too.