Guardian download

[Snowboaardvicky]

The Guardian download has failed on the last two days with the following errors.
(My backup of downloading The Independent won't work as the file size is too large for GMAIL)

Any thoughts and much thanks.

Fetch news from The Guardian and The Observer
Resolved conversion options
calibre version: 2.44.1
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'dehyphenate': True,
'delete_blank_paragraphs': True,
'disable_font_rescaling': False,
'dont_compress': False,
'dont_download_recipe': False,
'duplicate_links_in_toc': False,
'embed_all_fonts': False,
'embed_font_family': None,
'enable_heuristics': False,
'expand_css': False,
'extra_css': None,
'extract_to': None,
'filter_css': None,
'fix_indents': True,
'font_size_mapping': None,
'format_scene_breaks': True,
'html_unwrap_factor': 0.4,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x0290B6B0>,
'insert_blank_line': False,
'insert_blank_line_size': 0.5,
'insert_metadata': False,
'isbn': None,
'italicize_common_cases': True,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'markup_chapter_headings': True,
'max_toc_links': 50,
'minimum_line_height': 120.0,
'mobi_file_type': 'old',
'mobi_ignore_margins': False,
'mobi_keep_original_images': False,
'mobi_toc_at_start': False,
'no_chapters_in_toc': False,
'no_inline_navbars': True,
'no_inline_toc': False,
'output_profile': <calibre.customize.profiles.KindleOutput object at 0x0290BA10>,
'page_breaks_before': None,
'personal_doc': '[PDOC]',
'prefer_author_sort': False,
'prefer_metadata_cover': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_fake_margins': True,
'remove_first_image': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'renumber_headings': True,
'replace_scene_breaks': '',
'search_replace': None,
'series': None,
'series_index': None,
'share_not_sync': False,
'smarten_punctuation': False,
'sr1_replace': '',
'sr1_search': '',
'sr2_replace': '',
'sr2_search': '',
'sr3_replace': '',
'sr3_search': '',
'start_reading_at': None,
'subset_embedded_fonts': False,
'tags': None,
'test': False,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'toc_title': None,
'unsmarten_punctuation': False,
'unwrap_lines': True,
'use_auto_toc': False,
'verbose': 2}
InputFormatPlugin: Recipe Input running
Using custom recipe
Python function terminated unexpectedly
'NoneType' object has no attribute 'findAll' (Error Code: 1)
Traceback (most recent call last):
File "site.py", line 132, in main
File "site.py", line 109, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 193, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 25, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 1051, in run
File "site-packages\calibre\customize\conversion.py", line 241, in __call__
File "site-packages\calibre\ebooks\conversion\plugins\recipe_ input.py", line 117, in convert
File "site-packages\calibre\web\feeds\news.py", line 1029, in download
File "site-packages\calibre\web\feeds\news.py", line 1194, in build_index
File "<string>", line 166, in parse_index
File "<string>", line 132, in find_sections
AttributeError: 'NoneType' object has no attribute 'findAll'

[didsbury]

I'm having the same issues with both the Guardian and the Independent.

[gpauk]

Yes -- same problem yesterday and today! Hope there's a fix soon!

[Snowboaardvicky]

Thanks guys, I've had some issues with Comodo anti virus in the past blocking stuff and there had been a recent update to this which I'd installed. I didn't think it was on the same day as the Calibre issues started, but my memory isn't that good!

I'll wait and see if someone comes up with a solution.....I download The Scotsman in the end today which worked fine.

[gpauk]

This error crops up from time to time, if you google it. I suspect there's been some changes made to the websites... Hopefully a quick-ish fix...

[kovidgoyal]

https://github.com/kovidgoyal/calibr...64a9834b74950d

[gpauk]

Works a treat -- many thanks!

[didsbury]

First of all thanks for the very quick fix.

It works, but differently.

• Sections (including Sport) are missing
• The file size is larger (even with missing sections)

[kovidgoyal]

the website changed, now the recipe downloads whatever is on the todays paper page. http://www.guardian.co.uk/theguardian

[didsbury]

Is there a way to expand the coverage?

[kovidgoyal]

Sure you can have the recipe visit individual sections and get the articles from there, but you'd have to customize the recipe for that.

[kovidgoyal]

This shows you how to add sections to the recipe:

https://github.com/kovidgoyal/calibr...9c39b528639220

Simply add more lines like

feeds += self.parse_section('http://www.theguardian.com/uk/sport', 'Sport - ')

[didsbury]

Thanks, I'll have a go at that tomorrow.

[didsbury]

I have attached a file.

The paper is a different size every day, it's usually bigger on the weekends. However you can see that today's is much larger than last Friday's.

Also the section G2 is missing from today's download, this though does not appear on the front page of the website, so perhaps it can't be downloaded.

[didsbury]

Sorry, the file was too large to upload, I'll post it in Dropbox.