Thread: xkcd recipe
View Single Post
Old 01-20-2013, 04:25 PM   #1
RandomCake
Junior Member
RandomCake began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Jan 2011
Device: Kindle
xkcd recipe

Hi,
I've tried modifying the following line of the builtin XKCD recipe:
Code:
oldest_article = 60
and set it to:
Code:
oldest_article = 500
But, it only brings in 100 items (this is a one time request, to allow me to catch up, having not read the comic for ages).
Is there anything obvious I'm doing wrong?
The recipe in full is:
Code:
__license__   = 'GPL v3'
__copyright__ = '2008, Kovid Goyal <kovid at kovidgoyal.net>'
'''
Changelog:
2012-04-06
Fixed empty articles, added masthead img (NiLuJe)
2011-09-24
Changed cover (drMerry)
'''
'''
Fetch xkcd.
'''

import time, re
from calibre.web.feeds.news import BasicNewsRecipe

class XkcdCom(BasicNewsRecipe):
    cover_url = 'http://imgs.xkcd.com/static/terrible_small_logo.png'
    masthead_url = 'http://imgs.xkcd.com/static/terrible_small_logo.png'
    title = 'xkcd'
    description = 'A webcomic of romance and math humor.'
    __author__ = 'Martin Pitt updated by DrMerry.'
    language = 'en'

    use_embedded_content   = False
    oldest_article = 500
    #keep_only_tags = [dict(id='middleContainer')]
    #remove_tags = [dict(name='ul'), dict(name='h3'), dict(name='br')]
    keep_only_tags = [dict(id='comic')]
    no_stylesheets = True
    # turn image bubblehelp into a paragraph, and put alt in a heading
    preprocess_regexps = [
        (re.compile(r'(<img.*title=")([^"]+)(".alt=")([^"]+)(".*>)'),
         lambda m: '<h1>%s</h1>%s%s%s<p>%s</p>' % (m.group(4), m.group(1), m.group(3), m.group(5), m.group(2)))
    ]

    def parse_index(self):
        INDEX = 'http://xkcd.com/archive/'

        soup = self.index_to_soup(INDEX)
        articles = []
        for item in soup.findAll('a', title=True):
            articles.append({
                'date': item['title'],
                'timestamp': time.mktime(time.strptime(item['title'], '%Y-%m-%d'))+1,
                'url': 'http://xkcd.com' + item['href'],
                'title': self.tag_to_string(item),
                'description': '',
                'content': '',
            })

        return [('xkcd', articles)]
RandomCake is offline   Reply With Quote