Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 11-11-2010, 07:35 PM   #1
drgrittani
Junior Member
drgrittani began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Jul 2010
Location: Toronto, ON, Canada
Device: kobo
Recipe for a feed that posts epubs: Now Magazine Toronto

I searched and couldn't find the answer. I was trying to build a custom news recipe for a local Toronto Arts magazine (Now Magazine), and the feed (http://feeds.feedburner.com/NowEpubEditions) posts links to fully formatted epubs each week, so I would think it would be easy. I can load these epubs into Calibre fine from what has been posted, but I want to do it automatically. Any suggestions?
drgrittani is offline   Reply With Quote
Old 11-12-2010, 09:24 AM   #2
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by drgrittani View Post
the feed (http://feeds.feedburner.com/NowEpubEditions) posts links to fully formatted epubs each week, so I would think it would be easy. I can load these epubs into Calibre fine from what has been posted, but I want to do it automatically. Any suggestions?
The recipe system is not designed to do this. A website scraper is the normal solution. Wget, htttrack and web2disk may help.
Starson17 is offline   Reply With Quote
 
Enthusiast
Old 11-12-2010, 10:38 AM   #3
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,097
Karma: 5101571
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Actually with epub it can be done, look at the builtin taz digiabo recipe for example.
kovidgoyal is offline   Reply With Quote
Old 11-12-2010, 10:48 AM   #4
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by kovidgoyal View Post
Actually with epub it can be done, look at the builtin taz digiabo recipe for example.
Great. I'll do that. To the OP - sorry for the wrong info.
Starson17 is offline   Reply With Quote
Old 11-12-2010, 03:12 PM   #5
drgrittani
Junior Member
drgrittani began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Jul 2010
Location: Toronto, ON, Canada
Device: kobo
Thanks, I'll take a crack at it, I know a little programming, but nothing about python, hopefully I can just replace the url.
drgrittani is offline   Reply With Quote
Old 11-12-2010, 03:37 PM   #6
drgrittani
Junior Member
drgrittani began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Jul 2010
Location: Toronto, ON, Canada
Device: kobo
Starson17, it says you are a wizard so please post your results if you get it to work. Wizards can usually do these things more quickly.
drgrittani is offline   Reply With Quote
Old 11-14-2010, 12:27 PM   #7
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by drgrittani View Post
Starson17, it says you are a wizard so please post your results if you get it to work. Wizards can usually do these things more quickly.
This will grab the most recent EPUB on the feed (actually, the one nearest the top - which is the most recent). If you hadn't posted, and Kovid hadn't commented about the Taz Digiabo recipe, I never would have thought about doing it this way.

This recipe skips the parts of the recipe system that get the RSS feed, gets the masthead image and cover, parses the feed for articles, retrieves the articles, removes tags from articles, etc. All of those steps ultimately produce a local directory structure that looks just like an unzipped EPUB.

What this recipe does is to simply grab the link to one EPUB (the first in the RSS feed), saves the EPUB locally, extracts it, and passes the result back into the recipe system as though all the other steps had been completed normally. It was very interesting. (There's no easy way I see to combine multiple EPUBs from the feed.)
Spoiler:
Code:
#!/usr/bin/env  python
# -*- coding: utf-8 -*-
#Based on Lars Jacob's Taz Digiabo recipe

__license__   = 'GPL v3'
__copyright__ = '2010, Starson17'

import os, urllib2, zipfile
from calibre.web.feeds.news import BasicNewsRecipe
from calibre.ptempfile import PersistentTemporaryFile

class NowToronto(BasicNewsRecipe):
    title = u'Now Toronto'
    description = u'Now Toronto'
    __author__ = 'Starson17'
    conversion_options = {
        'no_default_epub_cover' : True
    }

    def build_index(self):
        epub_feed = "http://feeds.feedburner.com/NowEpubEditions"
        soup = self.index_to_soup(epub_feed)
        url = soup.find(name = 'feedburner:origlink').string
        f = urllib2.urlopen(url)
        tmp = PersistentTemporaryFile(suffix='.epub')
        self.report_progress(0,_('downloading epub'))
        tmp.write(f.read())
        tmp.close()
        zfile = zipfile.ZipFile(tmp.name, 'r')
        self.report_progress(0,_('extracting epub'))
        zfile.extractall(self.output_dir)
        tmp.close()
        index = os.path.join(self.output_dir, 'content.opf')
        self.report_progress(1,_('epub downloaded and extracted'))
        return index

Last edited by Starson17; 11-14-2010 at 12:34 PM.
Starson17 is offline   Reply With Quote
Old 11-14-2010, 09:51 PM   #8
drgrittani
Junior Member
drgrittani began at the beginning.
 
Posts: 9
Karma: 10
Join Date: Jul 2010
Location: Toronto, ON, Canada
Device: kobo
Thanks so much Starson17, works great.

Last edited by drgrittani; 11-14-2010 at 09:57 PM. Reason: I made a mistake, it did work
drgrittani is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Recipe for latest posts in mobileread forum oecherprinte Recipes 1 11-07-2010 09:49 PM
Create recipe for magazine BlonG Recipes 0 10-26-2010 07:46 AM
Custom Recipe for CNBC Magazine nittecat Calibre 1 02-28-2010 04:14 AM
Meet-Up! Toronto! Summer! Yay! (Toronto, Canada) ficbot Lounge 39 11-19-2009 09:39 PM


All times are GMT -4. The time now is 10:47 PM.


MobileRead.com is a privately owned, operated and funded community.