New York TImes recipe fails

[mikebw]

The recipe for The New York Times seems to fail at the point of downloading the latest recipe. Other recipes, such as The Washington Post, succeed.

calibre, version 3.44.0 (linux2, embedded-python: True)
Conversion error: Failed: Fetch news from The New York Times

Fetch news from The New York Times
Conversion options changed from defaults:
  output_profile: u'kindle'
  verbose: 2
Resolved conversion options
calibre version: 3.44.0
{'asciiize': False,
 'author_sort': None,
 'authors': None,
 'base_font_size': 0,
 'book_producer': None,
 'change_justification': u'original',
 'chapter': None,
 'chapter_mark': u'pagebreak',
 'comments': None,
 'cover': None,
 'debug_pipeline': None,
 'dehyphenate': True,
 'delete_blank_paragraphs': True,
 'disable_font_rescaling': False,
 'dont_compress': False,
 'dont_download_recipe': False,
 'duplicate_links_in_toc': False,
 'embed_all_fonts': False,
 'embed_font_family': None,
 'enable_heuristics': False,
 'expand_css': False,
 'extra_css': None,
 'extract_to': None,
 'filter_css': None,
 'fix_indents': True,
 'font_size_mapping': None,
 'format_scene_breaks': True,
 'html_unwrap_factor': 0.4,
 'input_encoding': None,
 'input_profile': <calibre.customize.profiles.InputProfile object at 0x7efdb2a822d0>,
 'insert_blank_line': False,
 'insert_blank_line_size': 0.5,
 'insert_metadata': False,
 'isbn': None,
 'italicize_common_cases': True,
 'keep_ligatures': False,
 'language': None,
 'level1_toc': None,
 'level2_toc': None,
 'level3_toc': None,
 'line_height': 0,
 'linearize_tables': False,
 'lrf': False,
 'margin_bottom': 5.0,
 'margin_left': 5.0,
 'margin_right': 5.0,
 'margin_top': 5.0,
 'markup_chapter_headings': True,
 'max_toc_links': 50,
 'minimum_line_height': 120.0,
 'mobi_file_type': u'old',
 'mobi_ignore_margins': False,
 'mobi_keep_original_images': False,
 'mobi_toc_at_start': False,
 'no_chapters_in_toc': False,
 'no_inline_navbars': True,
 'no_inline_toc': False,
 'output_profile': <calibre.customize.profiles.KindleOutput object at 0x7efdb2a82990>,
 'page_breaks_before': None,
 'personal_doc': u'[PDOC]',
 'prefer_author_sort': False,
 'prefer_metadata_cover': False,
 'pretty_print': False,
 'pubdate': None,
 'publisher': None,
 'rating': None,
 'read_metadata_from_opf': None,
 'remove_fake_margins': True,
 'remove_first_image': False,
 'remove_paragraph_spacing': False,
 'remove_paragraph_spacing_indent_size': 1.5,
 'renumber_headings': True,
 'replace_scene_breaks': u'',
 'search_replace': None,
 'series': None,
 'series_index': None,
 'share_not_sync': False,
 'smarten_punctuation': False,
 'sr1_replace': u'',
 'sr1_search': u'',
 'sr2_replace': u'',
 'sr2_search': u'',
 'sr3_replace': u'',
 'sr3_search': u'',
 'start_reading_at': None,
 'subset_embedded_fonts': False,
 'tags': None,
 'test': False,
 'timestamp': None,
 'title': None,
 'title_sort': None,
 'toc_filter': None,
 'toc_threshold': 6,
 'toc_title': None,
 'transform_css_rules': None,
 'unsmarten_punctuation': False,
 'unwrap_lines': True,
 'use_auto_toc': False,
 'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:nytimes_sub
Trying to get latest version of recipe: nytimes_sub
Traceback (most recent call last):
  File "site.py", line 77, in main
  File "site-packages/calibre/utils/ipc/worker.py", line 200, in main
  File "site-packages/calibre/gui2/convert/gui_conversion.py", line 35, in gui_convert_recipe
  File "site-packages/calibre/gui2/convert/gui_conversion.py", line 27, in gui_convert
  File "site-packages/calibre/ebooks/conversion/plumber.py", line 1110, in run
  File "site-packages/calibre/customize/conversion.py", line 246, in __call__
  File "site-packages/calibre/ebooks/conversion/plugins/recipe_input.py", line 139, in convert
  File "site-packages/calibre/web/feeds/news.py", line 1022, in download
  File "site-packages/calibre/web/feeds/news.py", line 1190, in build_index
  File "<string>", line 302, in parse_index
  File "<string>", line 206, in parse_todays_page
TypeError: 'NoneType' object has no attribute '__getitem__'

The web site of The New York Times is accessible normally in a web browser.

[kovidgoyal]

That's an error processing data from the NYT, probably just a temporary server glitch, try it again in a little while. I have also modified the recipe to ignore this type of invalid data rather than failing.

[mikebw]

It's working now but running unusually slowly, so your guess may be correct about a server problem.

Originally Posted by kovidgoyal

That's an error processing data from the NYT, probably just a temporary server glitch, try it again in a little while. I have also modified the recipe to ignore this type of invalid data rather than failing.