Custom recipes (archive, read-only)

[Sischa]

Originally Posted by kiklop74

I took a quick look, this site is really difficult for scraping. I'll see if I can do something during weekend though it is not a firm promise.

Thank you anyway. At least i am calmed now that it is not just me who has problems with the feed. I was a little bit confused why it doens't work on the first attemps but it came to my mind that i am just a little bit to stupid

I hope you'll maybe finde the time to look in this later cause its one of my favorite newspapers here in germany and their PDF epaper doesn't work in the kindle (a pitty but another story).

[johndoesecond]

Hi,

The NY Times (subscription) recipe doesn't seem to download Editorials and Op-Eds any more.
Does anyone else have this problem?

Thanks.

[Starson17]

I just posted a possible "bug" in the tracker, and it occurs to me that this recipe thread might be a good place to get more info on whether others see it as a bug, and/or whether my proposed solution might have problems. Here is my bug track post:

I have many books that have good author, title and series in the filenames, but poor metadata internally. When I'm adding them, I often need to turn on the Add/Save option "Get metadata only from filename." However, if I leave that option on, it takes me literally when recipes run, ignoring the good metadata a recipe provides. It appears to use a temporary filename for the recipe-based epub in the form:

appdata\local\temp\calibre_0.6.37_idcode_recipe_ou t.epub

I realize I can turn the option off off, but if I inadvertently leave it on overnight, or a recipe runs while I'm in the midst of adding books, I have to go back and fix those books or rerun the recipes. It's happened to me numerous times.

I don't know if you call this a bug or a feature, but I'm not sure why anyone would want this option to apply to recipe-based books. I only want it to apply to manually added books.

If you agree, here is a bit of code that prevents the option from applying to ebooks having a name that includes both 'calibre' and 'recipe.' AFAICT, all recipe based ebooks have those words in their temporary name, although I haven't checked this on all recipes or any other OS beyond Windows.

You might prefer some other test, even if you agree this change makes sense.

Does anyone else see this as a bug, or is there some reason why you would want the "Get metadata only from file name" option to apply to news recipes?

[newsfreak]

I'd like to get a recipe for New Straits Times (Malaysia) .
http://www.nst.com.my/

Thanks in advance.

[jpc66]

Hi,

Attached you will find the new version of Discover Magazine and Métro Montréal. These versions include the pictures related to the articles. I also cleaned the code up a little bit.

I also noticed that there is already a recipe for the Gazette called Montreal Gazette and made by Nick Redding. This is a better recipe than mine so I think you can delete my recipe. I don't see any reason to have two recipes for the same newspaper.

Regards,

JC

[PerErik87]

I found a list of all Nordic feeds. or most of them.
http://www.djh.dk/ejour/arkiv/RSS.html#Norge
If you want me to I can try to create recepies for them for you so you can add them easy. please tell me how you like them if you want me to do this.

Per Erik

[cypherslock]

There was a very nice gentleman who was working on a recipe that I requested: That of www.ps3center.net. I'd still like to have this if possible, and don't know how to do it myself otherwise I'd would.

[snlu178]

Originally Posted by lorenzov

try this one and customize the feeds as you wish

Thanks, this works great and was added in the new Calibre push. Much appreciated.

[JaClar]

Originally Posted by Baumi

My recipe wish is a little bit unusual in that I don't requite: The German newspaper taz already provides an epub edition (without DRM) for subscribers. When you go to http://www.taz.de/epub and enter valid credentials in the htaccess-form, the most current epub is automatically downloaded. To read it on my Cybook Gen3 with 1.5 firmware, I then use calibre to convert it, tag it as "News" and upload it to the reader.

...

Thanks for any infos.

I made a recipe for TAZ following the suggestions of kovidgoyal to rewrite the build_index() method:

#!/usr/bin/env  python
# -*- coding: utf-8 -*-

__license__   = 'GPL v3'
__copyright__ = '2010, Lars Jacob jacob.lars at gmail.com'
__docformat__ = 'restructuredtext de'

'''
www.taz.de/digiabo
'''
import os, re, urllib2, zipfile, tempfile
from calibre.web.feeds.news import BasicNewsRecipe

class TazDigiabo(BasicNewsRecipe):
	
	title = u'Taz Digiabo'
	description = u'Das EPUB DigiAbo der Taz'
	language = 'de'
	lang = 'de-DE'
	
	__author__ = 'Lars Jacob' 
	needs_subscription = True
	
	conversion_options = {
		'no_default_epub_cover' : True
	}
	
	def build_index(self):
		if self.username is not None and self.password is not None:
			domain = "http://www.taz.de"
			
			url = domain + "/digitaz/.digiabo"
			
			index = urllib2.urlopen(url)
			
			reg = "<a href=\"([^\"]*)\">taz_[0-9]{4}_[0-9]{2}_[0-9]{2}\.epub</a>"
			
			find = re.search(reg,index.read())
			
			issue = domain + find.group(1)
			
			auth_handler = urllib2.HTTPBasicAuthHandler()
			auth_handler.add_password(realm='TAZ-ABO',
									  uri=issue,
									  user=self.username,
									  passwd=self.password)
			opener = urllib2.build_opener(auth_handler)
			urllib2.install_opener(opener)
			
			try:
				f = urllib2.urlopen(issue)
			except urllib2.HTTPError as e:
				self.report_progress(0,_('Can\'t login to download %s.')%issue)
				return
			
			tmp = tempfile.TemporaryFile()
			self.report_progress(0,_('downloading epub'))
			tmp.write(f.read())
			
			zfile = zipfile.ZipFile(tmp, 'r')
			self.report_progress(0,_('extracting epub'))
			
			zfile.extractall(self.output_dir)
			
			tmp.close()
			index = os.path.join(self.output_dir, 'content.opf')
			
			self.report_progress(1,_('epub downloaded and extracted'))
			
			return index

I have to say that i'm a little bit underwhelmed with the result. Calibre reformats the whole book, which actually works quite well, but destroys the header quite a bit... Would be much nicer if Calibre supports the direct download of epub and other ebook files.

cheers,
jaclar

[for_give]

Adbusters Magazine?

www.Adbuster.org

I gave it a go and failed.

[PerErik87]

i ran into quite a few

usles divs
<!-- google_ad_section_start -->
thethingsineed
<!-- google_ad_section_end-->
usles divs
<!-- google_ad_section_start -->
thethingsineed
<!-- google_ad_section_end-->

How do i take advantage of these.
i think the answer is here somewhere.
http://docs.python.org/library/re.html#re-syntax

[LondoMolari]

Hi,

i tried Scinexx, a science magazine from german Springer Verlag. Its kinda raw, but more or less working...

from calibre.web.feeds.news import BasicNewsRecipe

class AdvancedUserRecipe1265145870(BasicNewsRecipe):
    title          = u'Scinexx.de'
    language = 'de'
    __author__ = 'JSuer'
    oldest_article = 14
    max_articles_per_feed = 100
    no_stylesheets = True

    feeds          = [(u'Scinexx.de', u'http://feeds.feedburner.com/scinexx')]

    remove_tags = [{'class':['text1fett']}]
    remove_tags = [{'href':['javascript:window.print()']}]

    def print_version(self, url):
        murxb = url.rfind('2010') - 6
        murxc = url[murxb :-5]
        murxx = 'http://www.scinexx.de/inc/artikel_drucken.php?id=' + murxc + '&a_flag=1'
        return murxx

[srvean]

Generic recipe
Do we have a generic recipe? Say I go to a blog site and I'm interested in an article which I want to store in my ebook reader. Currently what I do is Fetch News -> Add custom news source, create a new recipe using the GUI then go to Fetch news, and finally download the page. Too much work.

It would be wonderful if we had "Generic Recipe" - enter the URL at the Recipe and download the article; done.

Sort of what we would do using web2disk http://xxyy.com but would like to have this feature on the GUI.

[kiklop74]

Originally Posted by srvean

Generic recipe
Do we have a generic recipe? Say I go to a blog site and I'm interested in an article which I want to store in my ebook reader. Currently what I do is Fetch News -> Add custom news source, create a new recipe using the GUI then go to Fetch news, and finally download the page. Too much work.

It would be wonderful if we had "Generic Recipe" - enter the URL at the Recipe and download the article; done.

Sort of what we would do using web2disk http://xxyy.com but would like to have this feature on the GUI.

You can accomplish that task by using instapaper.com. Calibre has a recipe for that site. Go to the website, register and start adding articles you want to read. Once you are ready download them using calibre instapaper recipe. No coding involved at all.

[eolake]

Is there a recipe for TidBITS.com? (The world's longest-running and possibly best Macintosh newsletter.)