Economist failed after upgrade to 9.10

[kovidgoyal]

Originally Posted by NullNix

Yeah. Systems likely to experience ABI problems should probably have some scheme where you publicize the patches and let us point Calibre at a patched FF we built ourselves instead. (This might already be there: I haven't looked yet.)

(This is obviously unsatisfactory for those of us who don't already build our own FFs, but some of us do, or have distro tooling that can do that sort of thing trivially, like Nix (no relation)).

(and thanks for doing this!)

Patches are from the camoufox project not me. And yes at some point I will allow using distro provided camoufox.

[NullNix]

Originally Posted by kovidgoyal

Patches are from the camoufox project not me. And yes at some point I will allow using distro provided camoufox.

Aha, great! I can hack something ugly up until then

[kovidgoyal]

Originally Posted by NullNix

Aha, great! I can hack something ugly up until then

See https://github.com/kovidgoyal/calibr...875b6819391df0

[NullNix]

Originally Posted by kovidgoyal

See https://github.com/kovidgoyal/calibr...875b6819391df0

Fabulous, thank you! (And yes, Camoufox updates frequently, but honestly keeping up to date with rapidly changing stuff is something some distros actually *are* good at, and if you're using one of those... it's definitely not something Debian Stable users should do though.)

[eastonfmly]

Thank you so much Have a subscription but the ereader is so much easier Jim

[nickredding]

Originally Posted by kovidgoyal

Also I have no idea if it will continue to work once lots of people begin using it.

Absent IP address rotation I suspect it might not be a viable long-term solution.

[Cedrus]

Thank you so much. I did install .msi but download failed - detail of error below:

Spoiler:

calibre, version 9.15.100 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Economist

Fetch news from The Economist
Conversion options changed from defaults:
verbose: 2
output_profile: 'kindle_pw'
Resolved conversion options
calibre version: 9.15.100
{'add_alt_text_to_img': False,
'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_download_recipe': False,
'dont_split_on_page_breaks': True,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'epub_flatten': False,
'epub_inline_toc': False,
'epub_max_image_size': 'none',
'epub_toc_at_end': False,
'epub_version': '2',
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'flow_size': 260,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x000002A0C79523C0>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'no_chapters_in_toc': False,
'no_default_epub_cover': False,
'no_inline_navbars': False,
'no_svg_cover': False,
'output_profile': <calibre.customize.profiles.KindlePaperWhiteOutp ut object at 0x000002A0C7958D70>,
'page_breaks_before': None,
'prefer_metadata_cover': False,
'preserve_cover_aspect_ratio': False,
'pretty_print': True,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'recipe_specific_option': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'transform_html_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:economist
Trying to get latest version of recipe: economist
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36
Ignoring the requested user agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/148.0.0.0 Safari/537.36', camoufox uses one that matches the rest of its fingerprint
Warming up camoufox on: https://en.wikipedia.org/wiki/China, https://www.bbc.com/news/world
Camoufox ready in 57.2 seconds
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:152.0) Gecko/20100101 Firefox/152.0 Liskov
Recipe specific options:
res = 600
Traceback (most recent call last):
File "runpy.py", line 203, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 83, in <module>
File "site.py", line 78, in main
File "site.py", line 50, in run_entry_point
File "calibre\utils\ipc\worker.py", line 186, in main
File "calibre\gui2\convert\gui_conversion.py", line 49, in gui_convert_recipe
File "calibre\gui2\convert\gui_conversion.py", line 36, in gui_convert
File "calibre\ebooks\conversion\plumber.py", line 1258, in run
File "calibre\customize\conversion.py", line 242, in __call__
File "calibre\ebooks\conversion\plugins\recipe_input.py ", line 163, in convert
File "calibre\web\feeds\news.py", line 1189, in download
File "calibre\web\feeds\news.py", line 1359, in build_index
File "<string>", line 283, in parse_index
File "<string>", line 286, in economist_parse_index
File "calibre\web\feeds\news.py", line 792, in index_to_soup
File "calibre\web\automate\bot_check.py", line 65, in retrying
File "calibre\web\automate\recipes.py", line 733, in open_novisit
File "calibre\web\automate\recipes.py", line 713, in _open
urllib.error.URLError: <urlopen error HTTP 403>

[Cedrus]

Tried again this morning after installing version 9.15.101 - detail of error below:

Spoiler:

calibre, version 9.15.101 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Economist

Fetch news from The Economist
Conversion options changed from defaults:
output_profile: 'kindle_pw3'
verbose: 2
Resolved conversion options
calibre version: 9.15.101
{'add_alt_text_to_img': False,
'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_compress': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x0000013BD63AA660>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': 'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': True,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.KindlePaperWhite3Outpu t object at 0x0000013BD63B5160>,
'page_breaks_before': None,
'personal_doc': '[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'recipe_specific_option': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'transform_html_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:economist
Trying to get latest version of recipe: economist
Traceback (most recent call last):
File "runpy.py", line 203, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 83, in <module>
File "site.py", line 78, in main
File "site.py", line 50, in run_entry_point
File "calibre\utils\ipc\worker.py", line 186, in main
File "calibre\gui2\convert\gui_conversion.py", line 49, in gui_convert_recipe
File "calibre\gui2\convert\gui_conversion.py", line 36, in gui_convert
File "calibre\ebooks\conversion\plumber.py", line 1258, in run
File "calibre\customize\conversion.py", line 242, in __call__
File "calibre\ebooks\conversion\plugins\recipe_input.py ", line 162, in convert
File "calibre\web\feeds\news.py", line 1013, in __init__
ValueError: The "The Economist" recipe needs a username and password.

[kovidgoyal]

You need to have an eco account and subscription to use the recipe. If you have an account simply enter your eco username and password into the calibre fetch news dialog and it will work.