Recipe for a feed that posts epubs: Now Magazine Toronto

[drgrittani]

I searched and couldn't find the answer. I was trying to build a custom news recipe for a local Toronto Arts magazine (Now Magazine), and the feed (http://feeds.feedburner.com/NowEpubEditions) posts links to fully formatted epubs each week, so I would think it would be easy. I can load these epubs into Calibre fine from what has been posted, but I want to do it automatically. Any suggestions?

[Starson17]

Originally Posted by drgrittani

the feed (http://feeds.feedburner.com/NowEpubEditions) posts links to fully formatted epubs each week, so I would think it would be easy. I can load these epubs into Calibre fine from what has been posted, but I want to do it automatically. Any suggestions?

The recipe system is not designed to do this. A website scraper is the normal solution. Wget, htttrack and web2disk may help.

[kovidgoyal]

Actually with epub it can be done, look at the builtin taz digiabo recipe for example.

[Starson17]

Originally Posted by kovidgoyal

Actually with epub it can be done, look at the builtin taz digiabo recipe for example.

Great. I'll do that. To the OP - sorry for the wrong info.

[drgrittani]

Thanks, I'll take a crack at it, I know a little programming, but nothing about python, hopefully I can just replace the url.

[drgrittani]

Starson17, it says you are a wizard so please post your results if you get it to work. Wizards can usually do these things more quickly.

[Starson17]

Originally Posted by drgrittani

Starson17, it says you are a wizard so please post your results if you get it to work. Wizards can usually do these things more quickly.

This will grab the most recent EPUB on the feed (actually, the one nearest the top - which is the most recent). If you hadn't posted, and Kovid hadn't commented about the Taz Digiabo recipe, I never would have thought about doing it this way.

This recipe skips the parts of the recipe system that get the RSS feed, gets the masthead image and cover, parses the feed for articles, retrieves the articles, removes tags from articles, etc. All of those steps ultimately produce a local directory structure that looks just like an unzipped EPUB.

What this recipe does is to simply grab the link to one EPUB (the first in the RSS feed), saves the EPUB locally, extracts it, and passes the result back into the recipe system as though all the other steps had been completed normally. It was very interesting. (There's no easy way I see to combine multiple EPUBs from the feed.)

Spoiler:

#!/usr/bin/env  python
# -*- coding: utf-8 -*-
#Based on Lars Jacob's Taz Digiabo recipe

__license__   = 'GPL v3'
__copyright__ = '2010, Starson17'

import os, urllib2, zipfile
from calibre.web.feeds.news import BasicNewsRecipe
from calibre.ptempfile import PersistentTemporaryFile

class NowToronto(BasicNewsRecipe):
    title = u'Now Toronto'
    description = u'Now Toronto'
    __author__ = 'Starson17'
    conversion_options = {
        'no_default_epub_cover' : True
    }

    def build_index(self):
        epub_feed = "http://feeds.feedburner.com/NowEpubEditions"
        soup = self.index_to_soup(epub_feed)
        url = soup.find(name = 'feedburner:origlink').string
        f = urllib2.urlopen(url)
        tmp = PersistentTemporaryFile(suffix='.epub')
        self.report_progress(0,_('downloading epub'))
        tmp.write(f.read())
        tmp.close()
        zfile = zipfile.ZipFile(tmp.name, 'r')
        self.report_progress(0,_('extracting epub'))
        zfile.extractall(self.output_dir)
        tmp.close()
        index = os.path.join(self.output_dir, 'content.opf')
        self.report_progress(1,_('epub downloaded and extracted'))
        return index

[drgrittani]

Thanks so much Starson17, works great.