Wired Daily Edition now fails

[NSILMike]

Some articles download ok, others do this:

���r�H�0��z�2�۔V�DɔG��w|K�w�vIX ��E��y����3 ꒕����|x�ɛ���}�zQ�=�xx�0~u:̍؋�l��Ç��Y������C�p��41 ��ğ�����E�`�T��a���R��l�.�7�h��&�����9� ?�-ߥWB���fI��3�vL��}��h��R�\�+u�عkz�1���'�C����ܳ��o�� �n��v�K�qִ7{Mr��?�b��Z��.��M�9�K��р'}

for several pages...

[NSILMike]

Originally Posted by NSILMike

Some articles download ok, others do this:

���r�H�0��z�2�۔V�DɔG��w|K�w�vIX ��E��y����3 ꒕����|x�ɛ���}�zQ�=�xx�0~u:̍؋�l��Ç��Y������C�p��41 ��ğ�����E�`�T��a���R��l�.�7�h��&�����9� ?�-ߥWB���fI��3�vL��}��h��R�\�+u�عkz�1���'�C����ܳ��o�� �n��v�K�qִ7{Mr��?�b��Z��.��M�9�K��р'}

for several pages...

No one has any ideas why...? or how to fix?

[kovidgoyal]

Post the URL for a failing article.

[NSILMike]

I will when it happens again. It's not consistent day to day.
Thanks.

[NSILMike]

Hasn't happened again, most likely because the recipe now downloads almost nothing.
All that downloads is the cover and a page that used to list the various sections. Only one shows now: Top Stories. And it is always empty.

[kovidgoyal]

https://github.com/kovidgoyal/calibr...6819f8eae08f68

[NSILMike]

thanks! works much better (still skips one or two articles though.)

[NSILMike]

Now the recipe only downloads links.

[PatStapleton]

Seems they updated their feeds.

I've fixed those to point to the new feeds.
For some reason the Photo section articles don't load images correctly so I've left that feed disabled.

__license__ = 'GPL v3'
__copyright__ = '2014, Darko Miletic <darko.miletic at gmail.com>'
'''
www.wired.com
'''

from calibre import browser
from calibre.web.feeds.news import BasicNewsRecipe

class WiredDailyNews(BasicNewsRecipe):
    title = 'Wired Daily Edition'
    __author__ = 'Darko Miletic, PatStapleton(update 2020-05-24)'
    description = ('Wired is a full-color monthly American magazine, published in both print '
                   'and online editions, that reports on how emerging technologies affect culture,'
                   'the economy and politics.')
    masthead_url = 'https://www.wired.com/images/logos/apple-touch-icon.png'
    cover_url = 'https://www.wired.com/images/logos/wired.png'
    publisher = 'Conde Nast'
    category = 'news, IT, computers, technology'
    oldest_article = 2
    max_articles_per_feed = 200
    no_stylesheets = True
    encoding = 'utf-8'
    use_embedded_content = False
    language = 'en'
    ignore_duplicate_articles = {'url'}
    remove_empty_feeds = True
    publication_type = 'newsportal'
    extra_css             = """
                            .entry-header{
                                          text-transform: uppercase;
                                          vertical-align: baseline;
                                          display: inline;
                                         }
                            ul li{display: inline}
                            """

    remove_tags = [
		{
			'attrs': {
				'class': ['social-icons']
			}
		},
        dict(name=['meta', 'link', 'aside']),
        dict(id=['sharing', 'social', 'article-tags', 'sidebar']),
    ]
    keep_only_tags = [
        dict(name='article', attrs={'class': 'article main-content'}),
    ]
    remove_attributes = ['srcset']
    handle_gzip = True

    feeds = [
    (u'Top Stories', u'https://www.wired.com/feed/rss'),
    (u'Business', u'https://www.wired.com/feed/category/business/latest/rss'),
    (u'Culture', u'https://www.wired.com/feed/category/culture/latest/rss'),
    (u'Gear', u'https://www.wired.com/feed/category/gear/latest/rss'),
    (u'Ideas', u'https://www.wired.com/feed/category/ideas/latest/rss'),
    (u'Science', u'https://www.wired.com/feed/category/science/latest/rss'),
    (u'Security', u'https://www.wired.com/feed/category/security/latest/rss'),
    (u'Transportation', u'https://www.wired.com/feed/category/transportation/latest/rss'),
    (u'Backchannel', u'https://www.wired.com/feed/category/backchannel/latest/rss'),
    (u'WIRED Guides', u'https://www.wired.com/feed/tag/wired-guide/latest/rss'),
#    (u'Photo', u'https://www.wired.com/feed/category/photo/latest/rss'),
    ]

    def get_article_url(self, article):
        return article.get('link',  None)

    # Wired changes the content it delivers based on cookies, so the
    # following ensures that we send no cookies
    def get_browser(self, *args, **kwargs):
        return self

    def clone_browser(self, *args, **kwargs):
        return self.get_browser()

    def open_novisit(self, *args, **kwargs):
        br = browser()
        return br.open_novisit(*args, **kwargs)

    open = open_novisit