New York Times

[pdurrant]

Originally Posted by michailbachtin

On both my machines (Windows 10 & Windows 11) I get the same error message when trying to download the New York Times and the New York Times Book Review.

I'd be grateful for some help.

Version 8.10.0 doesn't have the fix in yet. No doubt there'll be a release soon with the fix included.

[PeterT]

Originally Posted by pdurrant

Version 8.10.0 doesn't have the fix in yet. No doubt there'll be a release soon with the fix included.

Actually that change was to a recipe that gets downloaded each and every time it's used.

[pdurrant]

Originally Posted by PeterT

Actually that change was to a recipe that gets downloaded each and every time it's used.

Oh! Fascinating. Thanks for the correction.

[mkgtu]

Originally Posted by PeterT

Actually that change was to a recipe that gets downloaded each and every time it's used.

So in other words, whatever "fix" was added isn't working. Needs another fix.
I just now for the second time today got the same error message about an expected delimiter:

Expecting ',' delimiter: line 1 column 564421 (char 564420)

Spoiler:

Fetch news from The New York Times
Conversion options changed from defaults:
output_profile: 'generic_eink_hd'
verbose: 2
Resolved conversion options
calibre version: 8.10.0
{'add_alt_text_to_img': False,
'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_download_recipe': False,
'dont_split_on_page_breaks': True,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'epub_flatten': False,
'epub_inline_toc': False,
'epub_max_image_size': 'none',
'epub_toc_at_end': False,
'epub_version': '2',
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'flow_size': 260,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x000001C4F8528610>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'no_chapters_in_toc': False,
'no_default_epub_cover': False,
'no_inline_navbars': False,
'no_svg_cover': False,
'output_profile': <calibre.customize.profiles.GenericEinkHD object at 0x000001C4F852BE50>,
'page_breaks_before': None,
'prefer_metadata_cover': False,
'preserve_cover_aspect_ratio': False,
'pretty_print': True,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'recipe_specific_option': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'transform_html_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:nytimes_sub
Trying to get latest version of recipe: nytimes_sub
Using user agent: User-Agent: Mozilla/5.0 (compatible; archive.org_bot; Wayback Machine Live Record; +http://archive.org/details/archive.org_bot)
Recipe specific options:
web = Todays Paper
days = 7
comp = no
Traceback (most recent call last):
File "runpy.py", line 198, in _run_module_as_main
File "runpy.py", line 88, in _run_code
File "site.py", line 83, in <module>
File "site.py", line 78, in main
File "site.py", line 50, in run_entry_point
File "calibre\utils\ipc\worker.py", line 213, in main
File "calibre\gui2\convert\gui_conversion.py", line 32, in gui_convert_recipe
File "calibre\gui2\convert\gui_conversion.py", line 26, in gui_convert
File "calibre\ebooks\conversion\plumber.py", line 1089, in run
File "calibre\customize\conversion.py", line 242, in __call__
File "calibre\ebooks\conversion\plugins\recipe_input.py ", line 153, in convert
File "calibre\web\feeds\news.py", line 1121, in download
File "calibre\web\feeds\news.py", line 1299, in build_index
File "<string>", line 223, in parse_index
File "<string>", line 197, in parse_todays_page
File "<string>", line 309, in parse_todays_page
File "<string>", line 269, in preloaded_data
File "json\decoder.py", line 353, in raw_decode
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 564421 (char 564420)

[unkn0wn]

https://github.com/kovidgoyal/calibr...d17fc7be9744b7

this was recently fixed but i think we would have to wait for the next release of calibre to use clean_js_json.

from calibre.web.site_parsers.nytimes import clean_js_json

I see that live update for nyt parser works when loading content for articles but not when importing clean_js_json.

from calibre.live import load_module

[kovidgoyal]

https://github.com/kovidgoyal/calibr...e7c782d8d2e2f8

[bhartman36]

The latest recipes I've been able to find on GitHub don't seem to work.

It seems like the NY Times is blocking fetches now.

mechanize._response.get_seek_wrapper_class.<locals >.httperror_seek_wrapper: HTTP Error 403: Not Allowed, Forbidden, Bot Blocked

[PeterT]

Originally Posted by bhartman36

The latest recipes I've been able to find on GitHub don't seem to work.

It seems like the NY Times is blocking fetches now.

See https://www.mobileread.com/forums/sh...d.php?t=370691