Instapaper - Updated recipe

[khromov]

I am maintaining an Instapaper recipe for Calibre together with Jim Ramsay. It allows fetching of your personal Instapaper news feed and uses Instapapers built-in text view for the best compatibility with e-readers.

There are two versions of the recipe – stable and development.

The stable version of the recipe is present in the main Calibre recipe branch. All you have to do is select the recipe in Calibre and it will automatically be kept up to date.

The development version introduced new features, such as automatically archiving items, folder support and multi-page article fetching. You can get this version from BitBucket.

Homepage:
http://khromov.wordpress.com/project...alibre-recipe/

Code hosting on Bitbucket. See link below:
https://bitbucket.org/khromov/calibre-instapaper/src

Download (latest version)
https://bitbucket.org/khromov/calibr...get/master.zip

[kovidgoyal]

Are you suggesting that this recipe should replace the builtin one?

[khromov]

kovid: Yes, this recipe is much better than the built-in one (which does not use the built-in text-only feature of Instapaper.)

[khromov]

Since I have you on the line kovid - any chance of building an online database of recipes? (Or is there one already I'm missing?)

That way people could post updates to recipes themselves, and you could vote for the best recipes for a given site. Maybe even an auto-update mechanism could get the latest recipes for Calibre automatically?

I'm a web developer and I'd love to help you with something like this. Let me know if you are interested.

[kovidgoyal]

recipes are executable code, having them update automatically from a repository to which anyone can make changes is not a good idea, to put it mildly.

All builtin recipes in calibre are updated updated automatically already, independent of calibre updates.

[khromov]

Originally Posted by kovidgoyal

recipes are executable code, having them update automatically from a repository to which anyone can make changes is not a good idea, to put it mildly.

All builtin recipes in calibre are updated updated automatically already, independent of calibre updates.

Recipes could be verified manually by an administrator (like you) before being added to the "official database" that Calibre uses, and the rest of the recipies (those awaiting official status and other versions for the same site) could be downloaded from the site manually by users.

Regarding the builtin recipes being updated - is there documentation on this? (Such as protocol, etc.)

[kovidgoyal]

Recipes are verified manually by me, that's how the builtin recipe updates work. When I commit an updated recipe to calibre's main development branch, it becomes available to all calibre installs automatically. There's no special protocol for it, just plain old http.

[Starson17]

The sticky here explains that builtins are automatically updated when they are used and that Kovid does the updating. A few days ago I added to the sticky suggesting that people clearly identify recipes here that they think should replace existing builtins. Kovid keeps a close eye on posts here. He picks up new recipes and changes to existing ones to add to the repository. It's helpful for each author to make it clear in his post when he thinks the changes are good for everyone (and should replace the builtin) or if they are just for a special case or special device.

[khromov]

Thank you both for the participation and information.

[ubieubie]

is there a limit to the number of articles that this recipe will send.
Thanks for the updated recipe

[Starson17]

Originally Posted by ubieubie

is there a limit to the number of articles that this recipe will send.
Thanks for the updated recipe

Yes - a year old and no more than 100 articles. You can change those in the recipe, but it's doubtful those are really the limits. The true limit will probably be the number of articles the recipe finds, and that's set by the site.

[khromov]

I have updated the recipe to fix a problem with Instapaper where article titles are not being displayed inside the article body.

The update recipe is in the first post.

[cendalc]

Thanks for the script. I had several minor issues with it:

1. wrong czech characters - fixed by forcing utf8 and embeding own fonts. Of course the font path is hardcoded according to my Nook.

    encoding                = 'utf8'
    extra_css               = '''
@font-face {
    font-style: italic;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: normal;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Italic.ttf');
}
@font-face {
    font-style: normal;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: normal;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Regular.ttf');
}
@font-face {
    font-style: italic;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: bold;
    src:url('res:///system/media/sdcard/my fonts/LiberationSerif-BoldItalic.ttf');
}
@font-face {
    font-style: normal;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: bold;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Bold.ttf');
}
body {
    font-family: 'LiberationSerif', serif;
}'''

2. It does not archive downloaded articles. I updated parse_index method to retrieve form key and added cleanup method:

    def parse_index(self):
        totalfeeds = []
        lfeeds = self.get_feeds()
        for feedobj in lfeeds:
            feedtitle, feedurl = feedobj
            self.report_progress(0, _('Fetching feed')+' %s...'%(feedtitle if feedtitle else feedurl))
            articles = []
            soup = self.index_to_soup(feedurl)
            self.myFormKey = soup.find('input', attrs={'name': 'form_key'})['value']
            for item in soup.findAll('div', attrs={'class':'cornerControls'}):
                description = self.tag_to_string(item.div)
                atag = item.a
                if atag and atag.has_key('href'):
                    url         = atag['href']
                    articles.append({
                                     'url'        :url
                                    })
            totalfeeds.append((feedtitle, articles))
        return totalfeeds

    def cleanup(self):
        params = urllib.urlencode(dict(form_key=self.myFormKey, submit="Archive All"))
        self.browser.open("http://www.instapaper.com/bulk-archive", params)

Thanks to banjopicker

3. Multipart articles in wrong order - fixed by

    reverse_article_order = True

[khromov]

Hey cendalc! Thanks for the input on the recipe. I'd love to implement some of your fixes, but I need some more information.

1.) Can you provide for me the link to a czech article so I can see if Kindle is affected by the encoding issue? Hopefully adding utf-8 as the encoding will be enough for most devices.

2.) Not archiving downloaded articles is by design, since reading old papers on a Kindle is a clunky process. This way you can archive the articles you read on the Instapaper site and get all your unread ones the next day. I'll see about adding an option flag for it though so users can pick their preferred behaviour.

3.) Can you please provide the link to one (or more) multipart articles that I can test the incorrect order behaviour on?

[cendalc]

Hi Zapt0,

1, 3) Here is a link to czech multipart article: http://www.diit.cz/clanek/mobilni-op...android/38512/ The list of content at the begining is still wrong (there is for example Mobilní instead of Mobilní) but the content itself is ok.

As to multipart - I just went through all chapters (kapitola) and added them to Instapaper manually - that's why I set reverse order. Do you somehow try to recognize multipart articles and download them at once or correct their order?

I also set reverse order because I like reading my articles in the same order they were added.

2) My use case is I download articles automatically everyday so I don't want to download the same articles that were downloaded yesterday.