Irish times recipe broken

[DavidBowie]

Hey, the irish times recipe is broken.

Hopefully it's just a simple fix? <3 to all OSS maintainers!!

Exception below:

Spoiler:

calibre, version 5.13.0 (win32, embedded-python: True)
Conversion error: Failed: Fetch news from The Irish Times

Fetch news from The Irish Times
Conversion options changed from defaults:
output_profile: 'tablet'
verbose: 2
Resolved conversion options
calibre version: 5.13.0
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_download_recipe': False,
'dont_split_on_page_breaks': True,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'epub_flatten': False,
'epub_inline_toc': False,
'epub_toc_at_end': False,
'epub_version': '2',
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'flow_size': 260,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x07BE8C88>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'no_chapters_in_toc': False,
'no_default_epub_cover': False,
'no_inline_navbars': False,
'no_svg_cover': False,
'output_profile': <calibre.customize.profiles.TabletOutput object at 0x07C07130>,
'page_breaks_before': None,
'prefer_metadata_cover': False,
'preserve_cover_aspect_ratio': False,
'pretty_print': True,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:irish_times
Trying to get latest version of recipe: irish_times
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36
Traceback (most recent call last):
File "runpy.py", line 194, in _run_module_as_main
File "runpy.py", line 87, in _run_code
File "site.py", line 82, in <module>
File "site.py", line 77, in main
File "site.py", line 49, in run_entry_point
File "calibre\utils\ipc\worker.py", line 216, in main
File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
File "calibre\ebooks\conversion\plumber.py", line 1105, in run
File "calibre\customize\conversion.py", line 245, in __call__
File "calibre\ebooks\conversion\plugins\recipe_input.py ", line 138, in convert
File "calibre\web\feeds\news.py", line 927, in __init__
File "<string>", line 73, in get_browser
TypeError: a bytes-like object is required, not 'str'

[kovidgoyal]

https://github.com/kovidgoyal/calibr...3ed9596f1e10a8

will fix this error, dont now if there are others, since i dont have a subscription.

[DavidBowie]

Ah, thanks so much! It authenticates correctly now, but I'm not getting any articles (weirdly enough the preview bit still works ok). I can set you up with a subscription to have a look? Are your PMs open?

[kovidgoyal]

Yes, you will need to pm me a working account details if you want me to have a look.

[DavidBowie]

PM sent

[kovidgoyal]

https://github.com/kovidgoyal/calibr...9db51f80cc6f6d

[DavidBowie]

As ever, you're the man

[quigo]

The Irish Times recipe is broken for me, I'm getting this error "ValueError: Failed to log in check username/password" I confirmed that my login details were working. Is it difficult to edit the recipe to get it working? I'm new to that but would like to understand what went wrong so I could fit it in the future.

[quigo]

The Irish Times recipe isn't working for me, below is the full error I'm getting: Is this difficult to fix as I'd like to understand how to be able to fix recipes in the future.

Spoiler:

Fetch news from The Irish Times
Conversion options changed from defaults:
output_profile: 'kindle_pw3'
verbose: 2
Resolved conversion options
calibre version: 5.34.0
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_compress': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x07517760>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': 'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': True,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.KindlePaperWhite3Outpu t object at 0x07517A60>,
'page_breaks_before': None,
'personal_doc': '[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'transform_css_rules': None,
'transform_html_rules': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
Failed to initialize plugin: 'C:\\Users\\daith\\AppData\\Roaming\\calibre\\plug ins\\DeDRM.zip'
InputFormatPlugin: Recipe Input running
Downloading recipe urn: builtin:irish_times
Trying to get latest version of recipe: irish_times
Using user agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.87 Safari/537.36
Traceback (most recent call last):
File "runpy.py", line 194, in _run_module_as_main
File "runpy.py", line 87, in _run_code
File "site.py", line 82, in <module>
File "site.py", line 77, in main
File "site.py", line 49, in run_entry_point
File "calibre\utils\ipc\worker.py", line 216, in main
File "calibre\gui2\convert\gui_conversion.py", line 34, in gui_convert_recipe
File "calibre\gui2\convert\gui_conversion.py", line 28, in gui_convert
File "calibre\ebooks\conversion\plumber.py", line 1111, in run
File "calibre\customize\conversion.py", line 244, in __call__
File "calibre\ebooks\conversion\plugins\recipe_input.py ", line 137, in convert
File "calibre\web\feeds\news.py", line 931, in __init__
File "<string>", line 74, in get_browser
ValueError: Failed to log in check username/password

[kovidgoyal]

That error says your username/password are not being accepted by the site. Are you sure they are correct? Can you login using them with a browser?

[quigo]

Yes I checked to make sure they worked and there was no problem. I'll send you them on DM.

[kovidgoyal]

Looks like the server is refusing to authenticate for some reason. I added some more header fields to make the http request look more like it is coming from a browser, which caused login to succeed once but then it went back to failing, some kind of bot protection I'd guess. I'm afraid I dont have the time/interest to figure out all the changes needed to convince the server to accept the request reliably. Here is my attempt: https://github.com/kovidgoyal/calibr...7c5237acd9efda

Feel free to play with it some more.

[quigo]

Thank you for your help, the recipe about a week ago started working apparently on its own..!