The Economist scheduled download

[Laforja]

The Economist has failed to download several times today. What is the solution? Calibre 8.5, following error message:

calibre, version 8.5.0 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Economist

Fetch news from The Economist
Conversion options changed from defaults:
verbose: 2
Resolved conversion options
calibre version: 8.5.0
{'add_alt_text_to_img': False,
'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_compress': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x000001E31B9EC9D0>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': 'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': False,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.OutputProfile object at 0x000001E31BA17250>,
'page_breaks_before': None,
'personal_doc': '[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'recipe_specific_option': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'transform_html_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:economist
Trying to get latest version of recipe: economist
Using user agent: Mozilla/5.0 (Linux; Android 14) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.6533.103 Mobile Safari/537.36 Liskov
Recipe specific options:
res = 600
de = No
Traceback (most recent call last):
File "runpy.py", line 198, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 83, in <module>
File "site.py", line 78, in main
File "site.py", line 50, in run_entry_point
File "calibre\utils\ipc\worker.py", line 213, in main
File "calibre\gui2\convert\gui_conversion.py", line 31, in gui_convert_recipe
File "calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
File "calibre\ebooks\conversion\plumber.py", line 1089, in run
File "calibre\customize\conversion.py", line 242, in __call__
File "calibre\ebooks\conversion\plugins\recipe_input.py ", line 153, in convert
File "calibre\web\feeds\news.py", line 1121, in download
File "calibre\web\feeds\news.py", line 1299, in build_index
File "<string>", line 343, in parse_index
File "<string>", line 550, in parse_web_index
File "calibre\web\feeds\news.py", line 752, in index_to_soup
File "mechanize\_mechanize.py", line 241, in open_novisit
File "mechanize\_mechanize.py", line 313, in _mech_open
mechanize._response.get_seek_wrapper_class.<locals >.httperror_seek_wrapper: HTTP Error 403: Forbidden

[yoyoisbest]

Same issue here

[econlerner]

Having the same issue. Any luck with a fix?

[InnovationHack]

Same and my morning read is now not happening.

[DNSB]

Do any of the people who posted in this thread have a subscription?

[Tlman46]

Same issue here, running in Linux.
It failed early today under 8.4, I updated to 8.5 and got the same issue.
Help!

Originally Posted by Laforja

The Economist has failed to download several times today. What is the solution? Calibre 8.5, following error message:

mechanize._response.get_seek_wrapper_class.<locals >.httperror_seek_wrapper: HTTP Error 403: Forbidden

[nickredding]

I don’t do calibre recipe development anymore, but for anyone who is interested the following works for the Economist weekly edition.

First, load https://www.economist.com/weeklyedition/archive

Pick out the anchor tags with href in the form /weeklyedition/yyyy-mm-dd

The anchor tags are in reverse-chronological order so the first one is the current edition, and subsequent tags are prior editions.

Now load https://www.economist.com/weeklyedition/yyyy-mm-dd. where yyyy-mm-dd is the edition date you want.

Now pick out the JSON object you can find inside <script id=“__NEXT DATA__”>, call this json (after decoding)

You can parse the edition contents by looking at json.props.pageProps.content.sections.articles

For all of these load (as well as articles) use user agent

Mozilla/5.0 (Linux; Android 14) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.6533.103 Mobile Safari/537.36 Lamarr

[joer2786]

Is this supposed to be for modifying the recipe or for some other way of managing?

[kovidgoyal]

Originally Posted by nickredding

I don’t do calibre recipe development anymore, but for anyone who is interested the following works for the Economist weekly edition.

First, load https://www.economist.com/weeklyedition/archive

Pick out the anchor tags with href in the form /weeklyedition/yyyy-mm-dd

The anchor tags are in reverse-chronological order so the first one is the current edition, and subsequent tags are prior editions.

Now load https://www.economist.com/weeklyedition/yyyy-mm-dd. where yyyy-mm-dd is the edition date you want.

Now pick out the JSON object you can find inside <script id=“__NEXT DATA__”>, call this json (after decoding)

You can parse the edition contents by looking at json.props.pageProps.content.sections.articles

For all of these load (as well as articles) use user agent

Mozilla/5.0 (Linux; Android 14) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.6533.103 Mobile Safari/537.36 Lamarr

Fixing the parsing of the index page is trivial, the problem is the eco no longer returns content on the actual article pages. Ideally for logged in users it would return content, but in some brief testing I did, I couldnt get it to work, have to look at it in more detail later.

You can see that only partial articles are available even in the wayback machine:
https://web.archive.org/web/20250620...israel-war-end

[yoyoisbest]

Originally Posted by kovidgoyal

Fixing the parsing of the index page is trivial, the problem is the eco no longer returns content on the actual article pages. Ideally for logged in users it would return content, but in some brief testing I did, I couldnt get it to work, have to look at it in more detail later.

You can see that only partial articles are available even in the wayback machine:
https://web.archive.org/web/20250620...israel-war-end

Would it be possible to create a beta version for testing? I have a subscription and would be happy to try it out and see if the full content loads correctly. That way, users with access could help verify if it’s working as expected.

[jwhittin]

Originally Posted by yoyoisbest

Would it be possible to create a beta version for testing? I have a subscription and would be happy to try it out and see if the full content loads correctly. That way, users with access could help verify if it’s working as expected.

+1 here

[kovidgoyal]

It isnt working so what would be the point of a beta version. If and when I can get something to work the recipe will be updated.

[Cough]

5.44 on 32 or 64 Bit windows is still working for the Economist.

[yoyoisbest]

Originally Posted by kovidgoyal

It isnt working so what would be the point of a beta version. If and when I can get something to work the recipe will be updated.

It works on commit 66a104e.
Thank you so much for all the work you’ve done!

[scottsan1]

Using Calibre 8.5 on my M3 iMac The Economist downloaded this morning in the normal fashion.