Le Monde Edition abonné does not work

[Muller]

Hello Kovid !

I've too been using a modified version of the recipe written years ago by Rémi Vanicat and I've had the error message "a unicode string is required". (It seems to happen only on tuesdays !)

Here is the full error message :

calibre, version 3.19.0 (win32, embedded-python: True)
Erreur lors de la conversion: Échoué: Récupérer des actualités à partir de Le Monde - Édition abonnés [20 mars 2018]

Récupérer des actualités à partir de Le Monde - Édition abonnés [20 mars 2018]
Conversion options changed from defaults:
verbose: 2
output_profile: u'kindle_voyage'
Resolved conversion options
calibre version: 3.19.0
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_compress': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x06183A30>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': 'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': True,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.KindleVoyageOutput object at 0x06183E30>,
'page_breaks_before': None,
'personal_doc': '[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Using custom recipe
Python function terminated unexpectedly
A unicode string is required (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 101, in main
File "site.py", line 78, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 195, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 26, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 1088, in run
File "site-packages\calibre\customize\conversion.py", line 245, in __call__
File "site-packages\calibre\ebooks\conversion\plugins\recipe_ input.py", line 119, in convert
File "site-packages\calibre\web\feeds\news.py", line 1018, in download
File "site-packages\calibre\web\feeds\news.py", line 1187, in build_index
File "site-packages\calibre\web\feeds\__init__.py", line 360, in feeds_from_index
File "site-packages\calibre\web\feeds\__init__.py", line 168, in populate_from_preparsed_feed
File "site-packages\calibre\web\feeds\__init__.py", line 23, in __init__
File "site-packages\calibre\utils\cleantext.py", line 17, in native_clean_xml_chars
TypeError: A unicode string is required


Thank you for your help !

[Muller]

Sorry for the long message. (I'm new to this.) Here is the recipe I'm using. Thanks.

[Phily]

Thanks Kovid!
I will do it.

[Phily]

It is just the case today. Here is the error log:
------------------------------------------
Récupérer des informations à partir de Le Monde: Édition abonnés papier
Conversion options changed from defaults:
output_profile: 'generic_eink'
verbose: 2
Resolved conversion options
calibre version: 3.15.0
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_download_recipe': False,
'dont_split_on_page_breaks': True,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'epub_flatten': False,
'epub_inline_toc': False,
'epub_toc_at_end': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'flow_size': 260,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x074320B0>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'no_chapters_in_toc': False,
'no_default_epub_cover': False,
'no_inline_navbars': False,
'no_svg_cover': False,
'output_profile': <calibre.customize.profiles.GenericEink object at 0x074322B0>,
'page_breaks_before': None,
'prefer_metadata_cover': False,
'preserve_cover_aspect_ratio': False,
'pretty_print': True,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Using custom recipe
Python function terminated unexpectedly
A unicode string is required (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 101, in main
File "site.py", line 78, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 195, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 26, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 1088, in run
File "site-packages\calibre\customize\conversion.py", line 245, in __call__
File "site-packages\calibre\ebooks\conversion\plugins\recipe_ input.py", line 119, in convert
File "site-packages\calibre\web\feeds\news.py", line 1023, in download
File "site-packages\calibre\web\feeds\news.py", line 1192, in build_index
File "site-packages\calibre\web\feeds\__init__.py", line 360, in feeds_from_index
File "site-packages\calibre\web\feeds\__init__.py", line 168, in populate_from_preparsed_feed
File "site-packages\calibre\web\feeds\__init__.py", line 23, in __init__
File "site-packages\calibre\utils\cleantext.py", line 17, in native_clean_xml_chars
TypeError: A unicode string is required

--------------
Do you need somthing else to solve the problem?

[kovidgoyal]

That error indicates that the title of one of the articles created by the recipe is not a string but instead None.

[Muller]

This week, the error occured today (monday)! Maybe it will tomorrow. Could someone with coding skills help with this "TypeError: A unicode string is required" problem ? I have tried to fix it, but failed completely. Thanks.

[Phily]

Reading Kovid advice, I have tried somethings with the titles. I am definitely unable. Sorry!

[kovidgoyal]

Well the next version of calibre will contain some code to just replace such invalid titles with 'Unknown'

[Muller]

Thank you, Kovid.

[Phily]

A lot of thanks, Kovid

[frague]

Thank you very much Kovid, it is working now ! I have tested the recipe on Calibre 3.18 (error message as expected) and on 3.21 (no error message).

It would be great to include this recipe by Remi Vanicat in the next release and with a different name. I suggest "Le Monde : édition abonnés papier" (it means Le Monde : subscriber paper edition).

Here is the working recipe I posted earlier.

[kovidgoyal]

https://github.com/kovidgoyal/calibr...6679a37000dcd5

[frague]

Originally Posted by kovidgoyal

https://github.com/kovidgoyal/calibr...6679a37000dcd5

Thank you very much Kovid ! This is great !

[Muller]

Hello ! It seems that the recipe "Le Monde Edition abonné", which had been corrected three months ago, may have problems once again.

It has been two or three weeks since I got on tuesday the error message "list index out of range" (no problem the other days).

More details attached.

Thanks for your help !

[kovidgoyal]

According to that you appear to be using a custom recipe. It refers to an error on line 117 of the recipe. The builtin recipe has only 102 lines.