Economist failed after upgrade to 9.10

[nilsws]

I've created a replacement that works for me, by letting calibre fetch through a real Chromium I already have installed, driven over the Chrome DevTools Protocol. Full write-up, setup steps and code:

https://github.com/CR0CKER/calibre-economist-cdp

Requirements: an Economist subscription, Chrome/Chromium, calibre 8 or 9, and Linux. No third-party Python packages.

Linux only: that is the only platform it has ever run on. Windows will probably trip over two permission-bit checks that assume POSIX semantics and over the "python3" interpreter name, and macOS may trip over a browser path containing spaces. Each is small and the specifics are in the repo. I have no Mac or Windows machine to test on, so I would welcome someone taking that on; pull requests welcome.

Setup is a one-time cookie seed (DevTools, Copy as cURL, import script, one command), after which scheduled downloads run unattended. Why it works, and why UA and cookie tricks do not, is in the README and in my reply in the thread above (post #N).

No downloaded editions are or will be shared here. Code only.

[johnny M]

when trying to download (economist magazine) fetch news with calibre program on windows 11 laptop
keep getting this error:

Spoiler:

calibre, version 9.14.0 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Economist

Fetch news from The Economist
Conversion options changed from defaults:
output_profile: 'generic_eink'
verbose: 2
Resolved conversion options
calibre version: 9.14.0
{'add_alt_text_to_img': False,
'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_download_recipe': False,
'dont_split_on_page_breaks': True,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'epub_flatten': False,
'epub_inline_toc': False,
'epub_max_image_size': 'none',
'epub_toc_at_end': False,
'epub_version': '2',
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'flow_size': 260,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x0000012EED439FD0>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'no_chapters_in_toc': False,
'no_default_epub_cover': False,
'no_inline_navbars': False,
'no_svg_cover': False,
'output_profile': <calibre.customize.profiles.GenericEink object at 0x0000012EED43B4D0>,
'page_breaks_before': None,
'prefer_metadata_cover': False,
'preserve_cover_aspect_ratio': False,
'pretty_print': True,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'recipe_specific_option': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'transform_html_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:economist
Trying to get latest version of recipe: economist
Using user agent: Mozilla/5.0 (Linux; Android 14) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.6533.103 Mobile Safari/537.36 Liskov
Recipe specific options:
res = 600
Failed to get index from website with error: HTTP Error 403: Forbidden
Falling back to GraphQL for the edition index
Failed to get index from GraphQL with error: <urlopen error [Errno 11001] getaddrinfo failed>
Traceback (most recent call last):
File "runpy.py", line 203, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 83, in <module>
File "site.py", line 78, in main
File "site.py", line 50, in run_entry_point
File "calibre\utils\ipc\worker.py", line 185, in main
File "calibre\gui2\convert\gui_conversion.py", line 49, in gui_convert_recipe
File "calibre\gui2\convert\gui_conversion.py", line 36, in gui_convert
File "calibre\ebooks\conversion\plumber.py", line 1258, in run
File "calibre\customize\conversion.py", line 242, in __call__
File "calibre\ebooks\conversion\plugins\recipe_input.py ", line 163, in convert
File "calibre\web\feeds\news.py", line 1152, in download
File "calibre\web\feeds\news.py", line 1322, in build_index
File "<string>", line 333, in parse_index
File "<string>", line 299, in economist_return_index
NoArticles: Could not find any articles, either the economist.com server is having trouble and you should try later or the website format has changed and the recipe needs to be updated.

[DNSB]

Removed unnecessary link after previous message moved to correct thread.