Custom recipes (archive, read-only)

[rty]

Originally Posted by capidamonte

Sadly, the Honolulu Advertiser was bought by the Honolulu Star-Bulletin. The new paper is called the Star Advertiser at www.staradvertiser.com.

Any chance of a new recipe? And the old ones are no longer valid...

Here it is: recipe for the Star Advertiser.

[capidamonte]

Originally Posted by rty

Here it is: recipe for the Star Advertiser.

Hey! Now there are two!

The most recent Calibre had a new one -- I'll have to try yours as well.

Thanks!

cap

[kidtwisted]

Hey Starson17, help!

Originally Posted by Starson17

Aren't recipes fun!



It's likely because of the order in which the various stages of the recipe are processed. I've certainly seen this. Once you get to the point where you are building your own pages from the soup (and that's what the multipage does) you don't get the expected behavior.

I believe the keep_only throws away the tags, during the initial page pull, but doesn't apply to the extra pages you are getting with the soup2 = self.index_to_soup(nexturl) step.

I've certainly seen this before. There are lots of solutions, in fact, your recipe already uses one - extract()- to remove a tag. Just find the tags and extract them.

I usually do this at the postprocess_html stage with something like this:

        for tag in soup.findAll('form', dict(attrs={'name':["comments_form"]})):
            tag.extract()
        for tag in soup.findAll('font', dict(attrs={'id':["cr-other-headlines"]})):
            tag.extract()

extract() removes the tag entirely from the original soup, leaving you with two independent soups. In your recipe, you want the extracted tag, but it also works to remove it from the original soup, just like remove_tags.

I've been having trouble making this work, adding this to the end of the recipe just breaks it. Can I get a more detailed example, I did read something about first_fetch but not sure how to use it. Is there another recipe I could look at for example?

    def postprocess_html(self, soup):
        for tag in soup.findAll('dic', dict(attrs={'class':["article-info clearfix"]})):
            tag.extract()
        return soup

[bhandarisaurabh]

Originally Posted by rty

Here it is: the recipe for Forbes India.

the recipe you have made does not cover whole magazine,it just fetches few of the feeds.Please do have a look at this link
http://business.in.com/magazine/magazinearchive/1/1
the recipe should download the latest issue from this link ,like in this case it should have fetched the 18 june issue.

[rty]

Originally Posted by bhandarisaurabh

the recipe you have made does not cover whole magazine,it just fetches few of the feeds.Please do have a look at this link
http://business.in.com/magazine/magazinearchive/1/1
the recipe should download the latest issue from this link ,like in this case it should have fetched the 18 june issue.

The recipe picks up articles from this link called "Forbes India All Feed"
(http://business.in.com/rssfeed/rss_all.xml) that supposedly contains all articles of the latest issue. The fact that the webmaster of Forbes India doesn't update the RSS feed to link all articles (as its name suggests) is beyond my comprehension and control but the recipe does point to the latest issue (whatever the date is). Yes, I went to the link you gave above. Please take a look again at the top right hand side of that webpage and find the column that says "Latest Issue" and let me know what you see.

Spoiler:

Click on the Show button above to see what I saw. You said 18 june is the latest issue but look at what the website said. It's 02 July.

[Starson17]

Originally Posted by kidtwisted

Hey Starson17, help!

I'll take a shot at it

I've been having trouble making this work, adding this to the end of the recipe just breaks it.

Does "this" refer to the code below? If so, try this:

    def postprocess_html(self, soup):
        for tag in soup.findAll('dic', dict(attrs={'class':["article-info clearfix"]})):
            #tag.extract()
            print 'The tag to be extracted is: ', tag
        return soup

If it's breaking because you're extracting something, then you probably shouldn't be extracting it - see what you're extracting with the print code above.

Can I get a more detailed example, I did read something about first_fetch but not sure how to use it. Is there another recipe I could look at for example?

The entirety of relevant code is in your example. You find the tag in the soup and extract it. I'm not sure what else to point you to.

[rty]

Originally Posted by Starson17

If it's breaking because you're extracting something, then you probably shouldn't be extracting it - see what you're extracting with the print code above.

Sorry to ask this noob question Starson. Where would you find the output of that print command?

I found out the hard way when I first used this tag.extract that "extract" under this context actually means "discard" or "dispose" or "get rid of".

[robandcurtis]

Can I get a custom recipe for the London Free Press at
http://www.lfpress.com/

Thanks!

[Starson17]

Originally Posted by rty

Sorry to ask this noob question Starson. Where would you find the output of that print command?

I always use ebook-convert recipename.recipe foldername --test -vv>recipename.txt during testing and it's in the .txt file. Alternatively, start the GUI with calibre-debug -g and it will appear there.

I found out the hard way when I first used this tag.extract that "extract" under this context actually means "discard" or "dispose" or "get rid of".

Not really. It means split it away from and make it independent of the soup. You end up with two pieces of soup. If you then don't use it, it's gone, but you are free to put it back someplace else. It makes sure that if you extract something, all traces of it are gone from the original soup, and all traces of the original soup are gone from it.

You were trying to do cleaning - extract is another way to do that.

[mlstein]

Can one of you generous and talented people revise the LRB recipe? I'm a subscriber and get all the web content, but the recipe gives me only a letter or two. Thanks!

Michael Steinberg

PS. calibre is da bomb!

[rty]

Originally Posted by robandcurtis

Can I get a custom recipe for the London Free Press at
http://www.lfpress.com/

Thanks!

I'll see what I can do this coming weekend. No promise though.

[robandcurtis]

Originally Posted by rty

I'll see what I can do this coming weekend. No promise though.

Thank you! No rush

[bhandarisaurabh]

Originally Posted by rty

The recipe picks up articles from this link called "Forbes India All Feed"
(http://business.in.com/rssfeed/rss_all.xml) that supposedly contains all articles of the latest issue. The fact that the webmaster of Forbes India doesn't update the RSS feed to link all articles (as its name suggests) is beyond my comprehension and control but the recipe does point to the latest issue (whatever the date is). Yes, I went to the link you gave above. Please take a look again at the top right hand side of that webpage and find the column that says "Latest Issue" and let me know what you see.

Spoiler:

Click on the Show button above to see what I saw. You said 18 june is the latest issue but look at what the website said. It's 02 July.

but if you go to the latest issue most of the links are not activated ,if you try to click the links the link says that the article will be available on 1st of july.Actually they are just giving you a fair idea of what is going to be in the latest issue beforehand but the latest issue is of18 june.

[rty]

Originally Posted by bhandarisaurabh

but if you go to the latest issue most of the links are not activated ,if you try to click the links the link says that the article will be available on 1st of july.Actually they are just giving you a fair idea of what is going to be in the latest issue beforehand but the latest issue is of18 june.

Look at the RSS page provided by Forbes India: http://business.in.com/rss/

As I mentioned, the recipe picks up articles from the feed called "Complete Business.in.com" http://business.in.com/rssfeed/rss_all.xml

Anything that is not included by Forbes India in this particular feed, there's nothing I can do about it. Maybe you can write to Forbes India to ask them to include all the articles of the latest issue in the RSS feed page and see if they care.

[rty]

Originally Posted by robandcurtis

Can I get a custom recipe for the London Free Press at
http://www.lfpress.com/

Thanks!

Here it is. Recipe for London Free Press (Canada).