Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 08-29-2011, 08:11 PM   #16
haroldtreen
Junior Member
haroldtreen began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Aug 2011
Device: Kindle 3
Quote:
Originally Posted by cendalc View Post
Thanks for the script. I had several minor issues with it:

1. wrong czech characters - fixed by forcing utf8 and embeding own fonts. Of course the font path is hardcoded according to my Nook.
Code:
    encoding                = 'utf8'
    extra_css               = '''
@font-face {
    font-style: italic;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: normal;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Italic.ttf');
}
@font-face {
    font-style: normal;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: normal;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Regular.ttf');
}
@font-face {
    font-style: italic;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: bold;
    src:url('res:///system/media/sdcard/my fonts/LiberationSerif-BoldItalic.ttf');
}
@font-face {
    font-style: normal;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: bold;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Bold.ttf');
}
body {
    font-family: 'LiberationSerif', serif;
}'''
2. It does not archive downloaded articles. I updated parse_index method to retrieve form key and added cleanup method:

Code:
    def parse_index(self):
        totalfeeds = []
        lfeeds = self.get_feeds()
        for feedobj in lfeeds:
            feedtitle, feedurl = feedobj
            self.report_progress(0, _('Fetching feed')+' %s...'%(feedtitle if feedtitle else feedurl))
            articles = []
            soup = self.index_to_soup(feedurl)
            self.myFormKey = soup.find('input', attrs={'name': 'form_key'})['value']
            for item in soup.findAll('div', attrs={'class':'cornerControls'}):
                description = self.tag_to_string(item.div)
                atag = item.a
                if atag and atag.has_key('href'):
                    url         = atag['href']
                    articles.append({
                                     'url'        :url
                                    })
            totalfeeds.append((feedtitle, articles))
        return totalfeeds

    def cleanup(self):
        params = urllib.urlencode(dict(form_key=self.myFormKey, submit="Archive All"))
        self.browser.open("http://www.instapaper.com/bulk-archive", params)
Thanks to banjopicker

3. Multipart articles in wrong order - fixed by
Code:
    reverse_article_order = True
These changes are awesome! I was looking for a way to archive articles I had already downloaded off Instapaper.

Thanks!
haroldtreen is offline   Reply With Quote
Old 09-15-2011, 08:22 AM   #17
sjchappy
Junior Member
sjchappy began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Sep 2011
Device: ST Nook
Is this new recipe working for people running the newest version of Calibre (0.8.18)?

If I try the recipe in the first thread post from zapt0 it runs fine, but only generates a three page document without any actual content; as if I had no articles in InstaPaper.

If I add in the changes from cendalc I get an error referencing 'self.myFormKey' in the 'cleanup' method. I also get this error using other recipes that add in this code in order to move articles to the archive. Is there anything special that needs to be done to use the 'form_key' attribute?

Any help would be appreciated.

Thanks.
sjchappy is offline   Reply With Quote
Old 09-16-2011, 09:39 AM   #18
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
Quote:
Originally Posted by sjchappy View Post
Is this new recipe working for people running the newest version of Calibre (0.8.18)?

If I try the recipe in the first thread post from zapt0 it runs fine, but only generates a three page document without any actual content; as if I had no articles in InstaPaper.

If I add in the changes from cendalc I get an error referencing 'self.myFormKey' in the 'cleanup' method. I also get this error using other recipes that add in this code in order to move articles to the archive. Is there anything special that needs to be done to use the 'form_key' attribute?

Any help would be appreciated.

Thanks.
Hey!

I have received my daily paper today without issues. Make sure you have your username and password set correctly. Also try using the built-in recipe in Calibre (it should be exactly the same as in my first post).

If you are still having issues, PM me your instapaper user/pass and I'll check it out.
khromov is offline   Reply With Quote
Old 09-18-2011, 05:35 PM   #19
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
Just a heads-up on the requested features:

Download all articles instead of just first page (20 articles)
This will be added in a future update.

Wrong czech fonts
Not possible to make all-round fix. Seems to be a Nook-specific issue, please use this code.

No auto-archive feature
I wish you could add as a checkbox under the recipe preferences for this but it is not possible and I don't use the feature. Please check here if you need it.

Multipart articles in wrong order
Simple reversing all the article order is not an option if you have many articles in your unread folder, but if you wish it is easy to add using this code in the beginning of the recipe:
Code:
reverse_article_order = True
I am using this recipe daily so I'll make sure to bring fixes as it breaks. Cheers!
khromov is offline   Reply With Quote
Old 10-04-2011, 10:29 PM   #20
jpcapili
Enthusiast
jpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notes
 
Posts: 38
Karma: 24968
Join Date: Sep 2011
Device: Kindle Touch 3G
the auto archive doesnt work. I've replaced the portion in the recipe with the code provided by banjopicker. the articles in my instapaper account are still unread. please advise.
jpcapili is offline   Reply With Quote
Old 10-05-2011, 04:39 AM   #21
cendalc
Junior Member
cendalc began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Jul 2011
Device: Nook
Quote:
Originally Posted by jpcapili View Post
the auto archive doesnt work. I've replaced the portion in the recipe with the code provided by banjopicker. the articles in my instapaper account are still unread. please advise.
You should replace it by my code. It works for me (Calibre 0.8.19).
cendalc is offline   Reply With Quote
Old 10-05-2011, 02:44 PM   #22
jpcapili
Enthusiast
jpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notesjpcapili can name that song in three notes
 
Posts: 38
Karma: 24968
Join Date: Sep 2011
Device: Kindle Touch 3G
Quote:
Originally Posted by cendalc View Post
You should replace it by my code. It works for me (Calibre 0.8.19).
replace the whole code or just that portion?
jpcapili is offline   Reply With Quote
Old 10-05-2011, 04:36 PM   #23
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
Quote:
Originally Posted by jpcapili View Post
replace the whole code or just that portion?
It's more like a merge operation...

1.) change the def parse_index(self) function chunk to cendalcs version.
2.) add the function def cleanup(self) function chunk somewhere inside the recipe.
khromov is offline   Reply With Quote
Old 10-05-2011, 10:03 PM   #24
LD B
Junior Member
LD B began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Oct 2011
Device: kindle
Instapaper/Livejournal issue

I saved livejournal posts to an instapaper folder, and now I'm trying to download the folder to Calibre. However, when I use the instapaper recipe (adding the code for the folder and multiple pages), the resulting ebook is missing text as some of the articles have livejournal notices in place of the actual post. So I wanted to download the text-only versions from instpaper, but when I use def print_version(self, url):
return 'http://www.instapaper.com' + url
I get none of the content. If anyone can tell what I'm doing wrong I would appreciate it. I have next to no experience with Python, so assume I know nothing and speak in very simple terms.
LD B is offline   Reply With Quote
Old 10-06-2011, 02:36 AM   #25
cendalc
Junior Member
cendalc began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Jul 2011
Device: Nook
Hey zapt0, maybe it would be better to add archive parameter to the class header, use my parse_index method and modify my cleanup:

Code:
class AdvancedUserRecipe1299694372(BasicNewsRecipe):
    title                             = u'Instapaper'
    archive                         = False

...

def parse_index(self):
        totalfeeds = []
        lfeeds = self.get_feeds()
        for feedobj in lfeeds:
            feedtitle, feedurl = feedobj
            self.report_progress(0, _('Fetching feed')+' %s...'%(feedtitle if feedtitle else feedurl))
            articles = []
            soup = self.index_to_soup(feedurl)
            self.myFormKey = soup.find('input', attrs={'name': 'form_key'})['value']
            for item in soup.findAll('div', attrs={'class':'cornerControls'}):
                description = self.tag_to_string(item.div)
                atag = item.a
                if atag and atag.has_key('href'):
                    url         = atag['href']
                    articles.append({
                                     'url'        :url
                                    })
            totalfeeds.append((feedtitle, articles))
        return totalfeeds

    def cleanup(self):
        if self.archive:
            params = urllib.urlencode(dict(form_key=self.myFormKey, submit="Archive All"))
            self.browser.open("http://www.instapaper.com/bulk-archive", params)
Users wanting auto-archive then change just one line:
Code:
archive = True
cendalc is offline   Reply With Quote
Old 10-11-2011, 05:09 PM   #26
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
Progress update: I've implemented all the requested features (article order reversal, multi-page article fetching, liked folder fetching, optional readability usage instead of Instapapers built-in text engine).

New version will be out in 3-5 days. Let me know if you want to test the development version for bugs, as I'm still ironing them out.
khromov is offline   Reply With Quote
Old 10-13-2011, 10:50 AM   #27
seasonedreader
Junior Member
seasonedreader began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Oct 2011
Device: Kindle
I'd be glad to test this preliminary version. I'm not a python developer but I think i can help.
seasonedreader is offline   Reply With Quote
Old 10-30-2011, 03:06 PM   #28
Erik van Oosten
Junior Member
Erik van Oosten began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Oct 2011
Device: Kindle 3
When there are no links in my instapaper account, I still get a new news document send to my kindle.

Is this a 'feature' of the instapaper recipe, or this in the recipe base code provided by Calibre?
If the latter is the case, I'll request for https://bugs.launchpad.net/bugs/883821 to be reopened.

Thanks and regards,
Erik.

PS. I'd be happy to test the preliminary version as well.
Erik van Oosten is offline   Reply With Quote
Old 10-30-2011, 06:11 PM   #29
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
I'd like for everyone to try out the new version of the recipe!

You can tweak a lot of settings right below the line:
Quote:
# - Recipe configuration - #
These settings include whether to include liked articles, auto-archive items, set number of maximal articles to fetch, reverse article order and more.

You can find the recipe at this link:
http://khromov.se/files/calibre/inst...calibre.recipe

Note: This recipe is still in testing and should not go into the main branch. Will post a stable version once this one undergoes more testing.

Note 2: Instapaper seems to be doing some rate-limiting and I do not recommend setting max_articles_per_feed to more than 100, or some articles may not fetch correctly.

Just let me know if you find any bugs, preferably with console output.

Quote:
Originally Posted by Erik van Oosten View Post
When there are no links in my instapaper account, I still get a new news document send to my kindle.
I'll need to look into it. Don't think it's a bug but I am unsure of best practices to stop execution of a recipe when there are no articles. Maybe someone more experienced can shed light on this?

Last edited by khromov; 10-30-2011 at 06:16 PM.
khromov is offline   Reply With Quote
Old 10-31-2011, 10:26 PM   #30
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,116
Karma: 5101571
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Quote:
Originally Posted by khromov View Post

I'll need to look into it. Don't think it's a bug but I am unsure of best practices to stop execution of a recipe when there are no articles. Maybe someone more experienced can shed light on this?
Simply return None from parse_index()
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Nature news - updated recipe Alexis Recipes 3 10-05-2012 02:36 PM
Recipe for National Geographic *Updated* gagsays Recipes 3 05-19-2011 12:30 PM
Updated Recipe: They Draw and Cook Starson17 Recipes 0 03-13-2011 10:59 AM
Updated recipe for Le Monde? veezh Recipes 5 01-20-2011 09:06 PM
One new recipe and other one updated (In Spanish) desUBIKado Recipes 3 01-19-2011 03:58 AM


All times are GMT -4. The time now is 05:45 AM.


MobileRead.com is a privately owned, operated and funded community.