Custom recipes (archive, read-only)

[fuquaforlife]

Hi!

Loving recipes for my ereader. Question -- is there a way to only download articles since the last time I downloaded? Or even (for example) parse through say the last few days of downloaded articles and toss out everything that is before the last time I downloaded? I hope that made sense

If this has been done before, perhaps someone can point me in the direction of a recipe where it has been done? Preferably of an RSS feed, since those are easier for me to decode and reverse engineer. I looked through a bunch of postings, but didn't see anything.

Thanks!!

[Starson17]

Originally Posted by fuquaforlife

Loving recipes for my ereader.

They are great aren't they!

Question -- is there a way to only download articles since the last time I downloaded?

That happens automatically - how could it d/l articles since before you last downloaded

Or even (for example) parse through say the last few days of downloaded articles and toss out everything that is before the last time I downloaded? I hope that made sense

If you just search for tag:news before you download, you can delete all the old news. If you've already d/l'd then search for something like date:>=yesterday or date:>=5DaysAgo


OOPS! I was thinking you wanted to control the old EPUBs, not the old articles. There is some talk of adding a control for this - keeping track of previously downloaded articles, but it's not yet implemented. I agree it would be useful.

[mwheinz]

Originally Posted by fuquaforlife

Hi!

Loving recipes for my ereader. Question -- is there a way to only download articles since the last time I downloaded? Or even (for example) parse through say the last few days of downloaded articles and toss out everything that is before the last time I downloaded? I hope that made sense

As far as I know, no - calibre doesn't compare the articles you're seeing in the rss feed with the articles you've already downloaded.

The only thing I could suggest is setting the limit to ignore articles more than 1 day old and then setting calibre to download new articles every day.

[kovidgoyal]

It's on the todo list.

[Starson17]

I seem to specialize in comics and food recipes. This is a bit of both! Food recipes in graphic form. Enjoy!

[Starson17]

Deleted as a dupe.

[Elevatorguy]

It would be great if someone could make a recipe for the show me city St. Louis Post-Dispatch & STLtoday.com, I use Instapaper to load individual articles, thanks.

[fuquaforlife]

Originally Posted by Starson17

They are great aren't they!

If you just search for tag:news before you download, you can delete all the old news. If you've already d/l'd then search for something like date:>=yesterday or date:>=5DaysAgo

OOPS! I was thinking you wanted to control the old EPUBs, not the old articles. There is some talk of adding a control for this - keeping track of previously downloaded articles, but it's not yet implemented. I agree it would be useful.

Thanks for quick response! I was afraid I wasn't being clear. Perhaps an example would be easier -- suppose I downloaded the news from the Washington Post on Wednesday at 2:30pm. And then the next day I downloaded the news from the Post 10am.

a) Side question: If I set it to download for the days = 1, does it download everything since Wednesday 12am? Or does days = 2 do that? And is it since 12am, or the current time?

b) What I was hoping to do originally was be able to download everything from Wednesday 12am, lift the value that I last downloaded at 2:30pm, and toss everything from before Wednesday 2:30pm.

Sounds like its on the the to do list though! Awesome work -- now I'm trying to get my 72-year old dad hooked on recipes so he doesn't have to read the news on his laptop.

[Starson17]

Originally Posted by fuquaforlife

a) Side question: If I set it to download for the days = 1, does it download everything since Wednesday 12am? Or does days = 2 do that? And is it since 12am, or the current time?

These are good questions. I've never looked closely enough to answer it. Why don't you test it and tell us all? I'd have to look at the code or test it to answer. I suspect it just compares the date, not the time, but that's just a guess. Personally, I never read every single recipe every day, so I like lots of overlap. I just read articles until I get to the ones I've already read.

Sounds like its on the the to do list though!

I believe that what is contemplated is to track the URL for each article. If that URL/article has been fetched previously, it gets skipped. AFAIK, it wouldn't try to keep track of a specific time, as that is already implemented in terms of the number of days to fetch.

[sark666]

I'm trying to make my first recipe but I can't see why this isn't working:

class AdvancedUserRecipe1280289069(BasicNewsRecipe):
    title          = u'Metro Daily'
    oldest_article = 7
    max_articles_per_feed = 100

    feeds          = [(u'metro local', u'http://www.metronews.ca/Toronto/local/rss')]

    def print_version(self, url):
        baseurl='http://www.metronews.ca/ArticlePrint/' 
        split1 = string.split(url,"/")
        xxx=split1 [6]
        split2= xxx[0:5]
        s =  baseurl + split2
        return s

It's from metronews, a Canadian daily news publication.

A url story would look like:

"http://www.metronews.ca/toronto/local/article/587467--finally-someone-at-the-acc-does-well"

But you can just have the 6 digit identifer. And the print equivalent would be:

"http://www.metronews.ca/ArticlePrint/587467?language=en"

But you only need the 6 digit identifer. At first I was adding the language bit, but when it wasn't working I removed it.

I tried it as str.split and string.split. Can't see where else it's wrong.

[Starson17]

Originally Posted by sark666

I'm trying to make my first recipe but I can't see why this isn't working:

Spoiler:

class AdvancedUserRecipe1280289069(BasicNewsRecipe):
    title          = u'Metro Daily'
    oldest_article = 7
    max_articles_per_feed = 100

    feeds          = [(u'metro local', u'http://www.metronews.ca/Toronto/local/rss')]

    def print_version(self, url):
        baseurl='http://www.metronews.ca/ArticlePrint/' 
        split1 = string.split(url,"/")
        xxx=split1 [6]
        split2= xxx[0:5]
        s =  baseurl + split2
        return s

It's from metronews, a Canadian daily news publication.

A url story would look like:

"http://www.metronews.ca/toronto/local/article/587467--finally-someone-at-the-acc-does-well"

But you can just have the 6 digit identifer. And the print equivalent would be:

"http://www.metronews.ca/ArticlePrint/587467?language=en"

But you only need the 6 digit identifer. At first I was adding the language bit, but when it wasn't working I removed it.

I tried it as str.split and string.split. Can't see where else it's wrong.

If you use string, you need to import it. You don't need it, however. Try:
split1 = url.split("/")
instead of
split1 = string.split(url,"/")

Also, check that your results are correct with a print, as in:
print 'split2 is: ', split2
right after you calc split2

[Venkman]

Hiya -

If someone would have the time to create a recipe for the Arizona Republic (http://www.azcentral.com/rss/feeds/news.xml), I would be grateful.

I've been trying to this myself, but after I get the idea of adding the feeds and adding the command "no_stylesheets = True", I don't understand what part of the article's/website's code does what to eliminate the unnecessary items or what commands eliminate which of these items.

I'm not a programmer and CSS just passed me by (I only know basic HTML). I don't mind if someone privately shares with me what to look for so I can learn how to do the job myself.

I've read the thread including the help documentation and it's like a foreign language to me - a flaw for sure.

Anyway, if anyone has the free time, thanks in advance. I'll try to figure some of this out as well.

[Starson17]

Originally Posted by Venkman

If someone would have the time to create a recipe for the Arizona Republic (http://www.azcentral.com/rss/feeds/news.xml), I would be grateful.

I've been trying to this myself, but after I get the idea of adding the feeds and adding the command "no_stylesheets = True", I don't understand what part of the article's/website's code does what to eliminate the unnecessary items or what commands eliminate which of these items.

I won't do it for you, but you sound like you're willing to learn, and I'll walk you through that - even if it takes longer

First, get FireFox or confirm you're using it and install FireBug. (needed to spot the stuff you want to remove when cleaning it up).

Second, post your current recipe (put Code then Spoiler tags around it).

We'll go from there.

[sark666]

If you use string, you need to import it. You don't need it, however. Try:
split1 = url.split("/")
instead of
split1 = string.split(url,"/")

Haivng limited knowledge of python (but learning) I'm not sure what you mean if I use string I need to import it. I'll try without.

You also mentioned to try print to make sure it's chopping up correctly, well I tried in a python editor previously and it's chopping up the string as I expected. I'm just not passing it back properly.

[Starson17]

Originally Posted by sark666

Haivng limited knowledge of python (but learning) I'm not sure what you mean if I use string I need to import it. I'll try without.

You would need this at the top of the recipe:

import string

Otherwise, your recipe doesn't know the string function.

You also mentioned to try print to make sure it's chopping up correctly, well I tried in a python editor previously and it's chopping up the string as I expected. I'm just not passing it back properly.

If you didn't have string imported, it would have failed. Didn't you get an error message? I like to print the results when the recipe runs, then I can see if it's doing what I expect. I take print statements out only after it's all working.