The Economist...

[jamescridland]

For me - the recipe (the one with The Economist logo on it) runs fine, and the first few articles display correctly. But everything from the letters page onward gives a paragraph or two followed by blurb asking me to sign up and talking about NewsGuard. Here's one example of an article - which for me looks like this (and looks similar on my Kobo Aura One).

Trying that article page in an incognito window, it looks like a different layout, with an older header, and different signup text (with no mention of NewsGuard).

(Trying again in an incognito window, I've been given the "NewsGuard" version).

I would therefore guess that they're testing a new version of their website, with different protection on it. I'd very much like a version of this recipe that I can log into.

[jamescridland]

I tried the other recipe (the one with the 'N' logo pointing downwards). That exhibits the same behaviour. Which looks like this, if you're interested.

[GGrams]

As of the Dec 6 issue of the Economist (Britain's Nightmare Before Christmas) I have the same problem with NewsGuard limiting full access to the articles and asking for me to subscribe. I've tried updating Calibre to the latest 4.5.0 version - to no avail. Is there anything I can do to get full access once again, other than paying a very expensive subscription fee?

[kovidgoyal]

As has been stated repeatedly in this thread all you need to do is retry the download a few times until you get the full version.

[Cedrus]

Hello,
I am new and am struggling to have full access to the latest issue of the Economist despite having downloaded version 4.5 of Calibre. I have tried several times already - to no avail. Can anyone explain what a recipe is ?
Many thanks & congratulations for creating Calibre

[kinming]

running calibre 2.75.1 inside chromebook, as a Linux app. Most recipes are fine, except Economist, which bombs, any idea? thanks;

calibre, version 2.75.1 (linux2, embedded-python: False)
Conversion Error: Failed: Fetch news from The Economist

Fetch news from The Economist
Resolved conversion options
calibre version: 2.75.1
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_compress': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x7a26f71b1d90>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': 'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': True,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.KindlePaperWhiteOutp ut object at 0x7a26f71b8550>,
'page_breaks_before': None,
'personal_doc': '[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Traceback (most recent call last):
File "/usr/bin/calibre-parallel", line 20, in <module>
sys.exit(main())
File "/usr/lib/calibre/calibre/utils/ipc/worker.py", line 195, in main
result = func(*args, **kwargs)
File "/usr/lib/calibre/calibre/gui2/convert/gui_conversion.py", line 26, in gui_convert
plumber.run()
File "/usr/lib/calibre/calibre/ebooks/conversion/plumber.py", line 1064, in run
accelerators, tdir)
File "/usr/lib/calibre/calibre/customize/conversion.py", line 245, in __call__
log, accelerators)
File "/usr/lib/calibre/calibre/ebooks/conversion/plugins/recipe_input.py", line 73, in convert
recipe = compile_recipe(self.recipe_source)
File "/usr/lib/calibre/calibre/web/feeds/recipes/__init__.py", line 48, in compile_recipe
exec src in namespace
File "<string>", line 11, in <module>
ImportError: No module named html5_parser

[aritafari]

I successfully downloaded the latest copy of The Economist.
Unlike last week's edition there were no calls to sign up. Thanks Kovid
Calibre 4.6

[kovidgoyal]

@kinming: Your version of calibre is too old for this recipe.

[epubli]

Dear Kovid,

I noticed that in the economist recipe, that some images retain their original width/height attributes which are much larger than the resolution of my E-Book reader. I downloaded The Economist using the Output-profile 'generic_eink'.

The effect is the following: Although the jpg images are scaled to a reasonable size e.g. 588 x 331 px by calibre, some of the low-resolution images are scaled up again to e.g. 1280 x 720 px by my E-Book reader due to CSS classes generated by calibre which specify the fixed height/width of the original images.

My solution was to remove all width/height attributes of all images by adding

        for img in soup.findAll('img', width=True):
            del img['width']
        for img in soup.findAll('img', height=True):
            del img['height']

to the postprocess_html. Maybe not the best solution, but I do not know in detail what went wrong during the conversion.

Now the images are displayed correctly. Thanks again for this great recipe !

[kovidgoyal]

https://github.com/kovidgoyal/calibr...fc06de84ec1be1

[tokjdm]

Is there an issue with this recipe ?

calibre, version 4.17.0 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Economist

Fetch news from The Economist
Conversion options changed from defaults:
verbose: 2
output_profile: 'kindle_pw'
Resolved conversion options
calibre version: 4.17.0
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': u'original',
'chapter': None,
'chapter_mark': u'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_compress': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x052FB270>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': u'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': True,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.KindlePaperWhiteOutp ut object at 0x052FB650>,
'page_breaks_before': None,
'personal_doc': u'[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': u'',
'search_replace': None,
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': u'',
'sr1_search': u'',
'sr2_replace': u'',
'sr2_search': u'',
'sr3_replace': u'',
'sr3_search': u'',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:economist
Trying to get latest version of recipe: economist
Using user agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36
Kindle Output profile being used, reducing image quality to keep file size below amazon email threshold
Python function terminated unexpectedly
Could not find any articles, either the economist.com server is having trouble and you should try later or the website format has changed and the recipe needs to be updated. (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 114, in main
File "site.py", line 88, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 209, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 36, in gui_convert_recipe
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 1110, in run
File "site-packages\calibre\customize\conversion.py", line 246, in __call__
File "site-packages\calibre\ebooks\conversion\plugins\recipe_ input.py", line 139, in convert
File "site-packages\calibre\web\feeds\news.py", line 1028, in download
File "site-packages\calibre\web\feeds\news.py", line 1196, in build_index
File "<string>", line 245, in parse_index
__builtin__.NoArticles: Could not find any articles, either the economist.com server is having trouble and you should try later or the website format has changed and the recipe needs to be updated.

[tokjdm]

I see that the recipe has been updated. Is there a way to copy the updated recipe into Calibre or should I wait for 4.17.X to arrive ?

[kovidgoyal]

you dont need to do anything recipes are auto-updated

[tokjdm]

Indeed it worked. Thank you very much.

[sup]

I have too computers, both in Czechia, both running Calibre 4.22. One is much slower but still usable for browsing and other basic stuff and when running the recipes, a lot of the articles beyond letters fail to download (they are missing completely). When run multiple times, different articles are missing. On the other computer, which is much faster, everything downloads just fine.

I logged on to the Economist website through my web browser on both as I am a subscriber - but if I understand it correctly, you do not need to be a subscriber to use this recipe to download the full version, so this should not make a difference, right?

Is it possible a computer might be too slow so that some articles fail to download? When I observed it in the job manager, it took about five seconds to download and article and when they started failing, it was taking much more, about one failure every 20-30 seconds. Any idea what might be causing this?