Custom recipes (archive, read-only)

[Starson17]

Originally Posted by gambarini

Because It is an obfuscated feed, ok...
otherwise the print link is under a findable tag.

Yes. It's not very obfuscated.

[Starson17]

Originally Posted by gambarini

i have solved my problem with parse index and with multiple pages...

Congratulations!

I "re"-post 3 recipes that i have posted but i don't find them in the new version of calibre.

Posting here makes it available to others faster. Posting in the bug tracker makes it more likely that Kovid will do something with the recipe - as it gets a number that he will definitely look at. When it's in the bug tracker, it's a strong sign that you consider it tested and ready for inclusion.

If it looks like you are still working on it, even after it is posted here, it probably won't be picked up. It's also easy to get lost here in all the posts, particularly if Kovid is busy on the code.

[rty]

Originally Posted by Starson17

So you don't need comments on why multipage didn't work when you copied from AdventureGamer?

Yes, I still am interested in learning why the multipage didn't work.

Thanks.

[Starson17]

Originally Posted by rty

How do we remove width property in tables? This conversion_options doesn't seem to remove it.

1. Sounds like a bug to me - ask for it to be fixed.
2. Meanwhile use preprocess_html, soup.findAll and extract from the soup all tags having attribute width (assuming only tags you want removed have that property), or
3. Use preprocess_regexps. It's basically text-based, not tag-based, search and replace in the html.

[Starson17]

Originally Posted by rty

Yes, I still am interested in learning why the multipage didn't work.

Thanks.

There are two major errors I see, before looking closely. First, there's no indent. (I'm sure you have it indented, so use the code tags to paste them in here. It's really hard to even read, much less test your recipe. If you really want to be nice to others, wrap code tags "#Button" around first, then wrap Spoiler "X on the Eye Button" around that.)

Second, I see append_page defined, but not used. You use it in preprocess_html.

Here is where it's used in AdventureGamers:

Spoiler:

    def preprocess_html(self, soup):
        mtag = '<meta http-equiv="Content-Language" content="en-US"/>\n<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>'
        soup.head.insert(0,mtag)    
        for item in soup.findAll(style=True):
            del item['style']
        self.append_page(soup, soup.body, 3)
        pager = soup.find('div',attrs={'class':'toolbar_fat'})
        if pager:
           pager.extract()        
        return soup

[rty]

Thanks Carson. I really appreciate your help. I have modified my original post to include code tag and spoiler button. Again thanks.

Spoiler:

 
class AdvancedUserRecipe1275708473(BasicNewsRecipe):
    title          = u'My Psychology Today'
    # oldest_article = 7
    max_articles_per_feed = 100
    remove_javascript = True
    use_embedded_content   = False
    no_stylesheets = True
    language = 'en'

    keep_only_tags = [dict(name='div', attrs={'id':['contentColumn','content-content']})]
    remove_tags = [
                    dict(name='div', attrs={'id':'advertisement advertisement-zone-51'}),
                    dict(name='div', attrs={'id':'block-td_search_160'}),
                    dict(name='div', attrs={'id':'block-cam_search_160'}),
                    dict(name='div', attrs={'class':'article-sub-meta'}),
   	dict(name='div', attrs={'class':'article-terms meta'}),
                         ] 
    # remove_tags_after  = dict(id=['rightColumn'])
    feeds          = [(u'Contents', u'http://www.psychologytoday.com/articles/index.rss')]

   
    def append_page(self, soup, appendtag, position):
        pager = soup.find('div',attrs={'class':'pager-next'})
        if pager:
           nexturl = self.INDEX + pager.a['href']
           soup2 = self.index_to_soup(nexturl)
           texttag = soup2.find('div', attrs={'id':'contentColumn'})
           for it in texttag.findAll(style=True):
               del it['style']
           newpos = len(texttag.contents)          
           self.append_page(soup2,texttag,newpos)
           texttag.extract()
           appendtag.insert(position,texttag)

    def postprocess_html(self, soup, first):
               for tag in soup.findAll(name=['ul', 'li']):
                    tag.name = 'div'
               return soup

[Starson17]

Originally Posted by rty

Thanks Carson. I really appreciate your help. I have modified my original post to include code tag and spoiler button. Again thanks.

It's Starson and you still haven't used append_page. Add preprocess_html the way that it's used in AG.

[rty]

Originally Posted by bhandarisaurabh

I would like to request a new recipe
for forbes india magazine
http://business.in.com/magazine/
thanks in advance

Here it is: the recipe for Forbes India.

[rty]

Originally Posted by Starson17

It's Starson and you still haven't used append_page. Add preprocess_html the way that it's used in AG.

Oops.... I am very sorry... please accept my apology, Starson.

Again thanks for the pointer. I'll study the Adventure Gamer code again.

[flyash]

Originally Posted by flyash

The current Instapaper recipe fetches all the articles in one's Instapaper account. Depending on one's browsing/reading habits, this often results in an ebook with articles on a variety of unrelated topics.

Instapaper has folders functionality, that allows you to group your articles however you see fit, for example, by subject matter. Can the Instapaper recipe be modified to fetch articles by folder - that way you end up with an ebook for each group of articles? And then users can use their ereader's functionality to categorize those ebooks as they normally would (e.g., use the Collections functionality in Sony readers).

Any interest in this? Or is it not technically feasible?

[rty]

Originally Posted by flyash

Any interest in this? Or is it not technically feasible?

What are you talking about? Instapaper has been incorporated into Calibre built-in recipes since long time ago.

[rty]

Attached is a recipe for Today Online - Another Singapore English Daily Newspaper

[flyash]

Originally Posted by rty

What are you talking about? Instapaper has been incorporated into Calibre built-in recipes since long time ago.

I'm not sure how I would explain it differently than in the post above, but I'll try. The current Instapaper recipe fetches all unarchived articles in your Instapaper account. But now that Instapaper has folders, I'm wondering if the recipe can be modified so that it fetches articles by folder, that way you create a different ebook for each Instapaper folder's articles (that way your articles are organized by subject, rather than having one big ebook with an assortment of articles on various subjects).

Currently, if you put all your Instapaper articles in folders, the Instapaper recipe fetches nothing, so apparently it can only see unfoldered articles.

And I'm not suggesting replacing the old Instapaper recipe (since some people with only a few articles might prefer not using Instapaper's folders and just having everything in one ebook), but adding a new recipe(s) that does a folder fetch.

[Starson17]

Originally Posted by rty

Oops.... I am very sorry... please accept my apology, Starson.

Again thanks for the pointer. I'll study the Adventure Gamer code again.

No problem
Let us know if you have trouble with the AG code.

[flyash]

Originally Posted by flyash

Currently, if you put all your Instapaper articles in folders, the Instapaper recipe fetches nothing, so apparently it can only see unfoldered articles.

It seems that Instapaper defaults to putting saved articles in a 'Read Later' folder, and this is what Calibre's recipe is looking for, so if you make new folders to organize your articles and move all unarchived articles to the new folders, the recipe will only fetch from the empty 'Read Later' folder, and return an empty ebook.

So it would seem that Calibre's recipe could be modified to look for folders other than 'Read Later'. And maybe the simplest solution is for users to use custom recipes, one for each Instapaper folder.