Instapaper - Updated recipe

[haroldtreen]

Originally Posted by cendalc

Thanks for the script. I had several minor issues with it:

1. wrong czech characters - fixed by forcing utf8 and embeding own fonts. Of course the font path is hardcoded according to my Nook.

    encoding                = 'utf8'
    extra_css               = '''
@font-face {
    font-style: italic;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: normal;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Italic.ttf');
}
@font-face {
    font-style: normal;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: normal;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Regular.ttf');
}
@font-face {
    font-style: italic;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: bold;
    src:url('res:///system/media/sdcard/my fonts/LiberationSerif-BoldItalic.ttf');
}
@font-face {
    font-style: normal;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: bold;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Bold.ttf');
}
body {
    font-family: 'LiberationSerif', serif;
}'''

2. It does not archive downloaded articles. I updated parse_index method to retrieve form key and added cleanup method:

    def parse_index(self):
        totalfeeds = []
        lfeeds = self.get_feeds()
        for feedobj in lfeeds:
            feedtitle, feedurl = feedobj
            self.report_progress(0, _('Fetching feed')+' %s...'%(feedtitle if feedtitle else feedurl))
            articles = []
            soup = self.index_to_soup(feedurl)
            self.myFormKey = soup.find('input', attrs={'name': 'form_key'})['value']
            for item in soup.findAll('div', attrs={'class':'cornerControls'}):
                description = self.tag_to_string(item.div)
                atag = item.a
                if atag and atag.has_key('href'):
                    url         = atag['href']
                    articles.append({
                                     'url'        :url
                                    })
            totalfeeds.append((feedtitle, articles))
        return totalfeeds

    def cleanup(self):
        params = urllib.urlencode(dict(form_key=self.myFormKey, submit="Archive All"))
        self.browser.open("http://www.instapaper.com/bulk-archive", params)

Thanks to banjopicker

3. Multipart articles in wrong order - fixed by

    reverse_article_order = True

These changes are awesome! I was looking for a way to archive articles I had already downloaded off Instapaper.

Thanks!

[sjchappy]

Is this new recipe working for people running the newest version of Calibre (0.8.18)?

If I try the recipe in the first thread post from zapt0 it runs fine, but only generates a three page document without any actual content; as if I had no articles in InstaPaper.

If I add in the changes from cendalc I get an error referencing 'self.myFormKey' in the 'cleanup' method. I also get this error using other recipes that add in this code in order to move articles to the archive. Is there anything special that needs to be done to use the 'form_key' attribute?

Any help would be appreciated.

Thanks.

[khromov]

Originally Posted by sjchappy

Is this new recipe working for people running the newest version of Calibre (0.8.18)?

If I try the recipe in the first thread post from zapt0 it runs fine, but only generates a three page document without any actual content; as if I had no articles in InstaPaper.

If I add in the changes from cendalc I get an error referencing 'self.myFormKey' in the 'cleanup' method. I also get this error using other recipes that add in this code in order to move articles to the archive. Is there anything special that needs to be done to use the 'form_key' attribute?

Any help would be appreciated.

Thanks.

Hey!

I have received my daily paper today without issues. Make sure you have your username and password set correctly. Also try using the built-in recipe in Calibre (it should be exactly the same as in my first post).

If you are still having issues, PM me your instapaper user/pass and I'll check it out.

[khromov]

Just a heads-up on the requested features:

Download all articles instead of just first page (20 articles)
This will be added in a future update.

Wrong czech fonts
Not possible to make all-round fix. Seems to be a Nook-specific issue, please use this code.

No auto-archive feature
I wish you could add as a checkbox under the recipe preferences for this but it is not possible and I don't use the feature. Please check here if you need it.

Multipart articles in wrong order
Simple reversing all the article order is not an option if you have many articles in your unread folder, but if you wish it is easy to add using this code in the beginning of the recipe:

reverse_article_order = True

I am using this recipe daily so I'll make sure to bring fixes as it breaks. Cheers!

[jpcapili]

the auto archive doesnt work. I've replaced the portion in the recipe with the code provided by banjopicker. the articles in my instapaper account are still unread. please advise.

[cendalc]

Originally Posted by jpcapili

the auto archive doesnt work. I've replaced the portion in the recipe with the code provided by banjopicker. the articles in my instapaper account are still unread. please advise.

You should replace it by my code. It works for me (Calibre 0.8.19).

[jpcapili]

Originally Posted by cendalc

You should replace it by my code. It works for me (Calibre 0.8.19).

replace the whole code or just that portion?

[khromov]

Originally Posted by jpcapili

replace the whole code or just that portion?

It's more like a merge operation...

1.) change the def parse_index(self) function chunk to cendalcs version.
2.) add the function def cleanup(self) function chunk somewhere inside the recipe.

[LD B]

I saved livejournal posts to an instapaper folder, and now I'm trying to download the folder to Calibre. However, when I use the instapaper recipe (adding the code for the folder and multiple pages), the resulting ebook is missing text as some of the articles have livejournal notices in place of the actual post. So I wanted to download the text-only versions from instpaper, but when I use def print_version(self, url):
return 'http://www.instapaper.com' + url
I get none of the content. If anyone can tell what I'm doing wrong I would appreciate it. I have next to no experience with Python, so assume I know nothing and speak in very simple terms.

[cendalc]

Hey zapt0, maybe it would be better to add archive parameter to the class header, use my parse_index method and modify my cleanup:

class AdvancedUserRecipe1299694372(BasicNewsRecipe):
    title                             = u'Instapaper'
    archive                         = False

...

def parse_index(self):
        totalfeeds = []
        lfeeds = self.get_feeds()
        for feedobj in lfeeds:
            feedtitle, feedurl = feedobj
            self.report_progress(0, _('Fetching feed')+' %s...'%(feedtitle if feedtitle else feedurl))
            articles = []
            soup = self.index_to_soup(feedurl)
            self.myFormKey = soup.find('input', attrs={'name': 'form_key'})['value']
            for item in soup.findAll('div', attrs={'class':'cornerControls'}):
                description = self.tag_to_string(item.div)
                atag = item.a
                if atag and atag.has_key('href'):
                    url         = atag['href']
                    articles.append({
                                     'url'        :url
                                    })
            totalfeeds.append((feedtitle, articles))
        return totalfeeds

    def cleanup(self):
        if self.archive:
            params = urllib.urlencode(dict(form_key=self.myFormKey, submit="Archive All"))
            self.browser.open("http://www.instapaper.com/bulk-archive", params)

Users wanting auto-archive then change just one line:

archive = True

[khromov]

Progress update: I've implemented all the requested features (article order reversal, multi-page article fetching, liked folder fetching, optional readability usage instead of Instapapers built-in text engine).

New version will be out in 3-5 days. Let me know if you want to test the development version for bugs, as I'm still ironing them out.

[seasonedreader]

I'd be glad to test this preliminary version. I'm not a python developer but I think i can help.

[Erik van Oosten]

When there are no links in my instapaper account, I still get a new news document send to my kindle.

Is this a 'feature' of the instapaper recipe, or this in the recipe base code provided by Calibre?
If the latter is the case, I'll request for https://bugs.launchpad.net/bugs/883821 to be reopened.

Thanks and regards,
Erik.

PS. I'd be happy to test the preliminary version as well.

[khromov]

I'd like for everyone to try out the new version of the recipe!

You can tweak a lot of settings right below the line:

# - Recipe configuration - #

These settings include whether to include liked articles, auto-archive items, set number of maximal articles to fetch, reverse article order and more.

You can find the recipe at this link:
http://khromov.se/files/calibre/inst...calibre.recipe

Note: This recipe is still in testing and should not go into the main branch. Will post a stable version once this one undergoes more testing.

Note 2: Instapaper seems to be doing some rate-limiting and I do not recommend setting max_articles_per_feed to more than 100, or some articles may not fetch correctly.

Just let me know if you find any bugs, preferably with console output.

Originally Posted by Erik van Oosten

When there are no links in my instapaper account, I still get a new news document send to my kindle.

I'll need to look into it. Don't think it's a bug but I am unsure of best practices to stop execution of a recipe when there are no articles. Maybe someone more experienced can shed light on this?

[kovidgoyal]

Originally Posted by khromov

I'll need to look into it. Don't think it's a bug but I am unsure of best practices to stop execution of a recipe when there are no articles. Maybe someone more experienced can shed light on this?

Simply return None from parse_index()