Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 07-18-2011, 03:30 PM   #1
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
Instapaper - Updated recipe

I am maintaining an Instapaper recipe for Calibre together with Jim Ramsay. It allows fetching of your personal Instapaper news feed and uses Instapapers built-in text view for the best compatibility with e-readers.

There are two versions of the recipe – stable and development.

The stable version of the recipe is present in the main Calibre recipe branch. All you have to do is select the recipe in Calibre and it will automatically be kept up to date.

The development version introduced new features, such as automatically archiving items, folder support and multi-page article fetching. You can get this version from BitBucket.

Homepage:
http://khromov.wordpress.com/project...alibre-recipe/

Code hosting on Bitbucket. See link below:
https://bitbucket.org/khromov/calibre-instapaper/src

Download (latest version)
https://bitbucket.org/khromov/calibr...get/master.zip

Last edited by khromov; 06-01-2012 at 06:40 PM.
khromov is offline   Reply With Quote
Old 07-18-2011, 03:59 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Are you suggesting that this recipe should replace the builtin one?
kovidgoyal is offline   Reply With Quote
 
Enthusiast
Old 07-18-2011, 05:28 PM   #3
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
kovid: Yes, this recipe is much better than the built-in one (which does not use the built-in text-only feature of Instapaper.)
khromov is offline   Reply With Quote
Old 07-18-2011, 05:46 PM   #4
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
Since I have you on the line kovid - any chance of building an online database of recipes? (Or is there one already I'm missing?)

That way people could post updates to recipes themselves, and you could vote for the best recipes for a given site. Maybe even an auto-update mechanism could get the latest recipes for Calibre automatically?

I'm a web developer and I'd love to help you with something like this. Let me know if you are interested.
khromov is offline   Reply With Quote
Old 07-18-2011, 06:42 PM   #5
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
recipes are executable code, having them update automatically from a repository to which anyone can make changes is not a good idea, to put it mildly.

All builtin recipes in calibre are updated updated automatically already, independent of calibre updates.
kovidgoyal is offline   Reply With Quote
Old 07-19-2011, 04:21 AM   #6
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
Quote:
Originally Posted by kovidgoyal View Post
recipes are executable code, having them update automatically from a repository to which anyone can make changes is not a good idea, to put it mildly.

All builtin recipes in calibre are updated updated automatically already, independent of calibre updates.
Recipes could be verified manually by an administrator (like you) before being added to the "official database" that Calibre uses, and the rest of the recipies (those awaiting official status and other versions for the same site) could be downloaded from the site manually by users.

Regarding the builtin recipes being updated - is there documentation on this? (Such as protocol, etc.)
khromov is offline   Reply With Quote
Old 07-19-2011, 08:56 AM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,154
Karma: 5381911
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Recipes are verified manually by me, that's how the builtin recipe updates work. When I commit an updated recipe to calibre's main development branch, it becomes available to all calibre installs automatically. There's no special protocol for it, just plain old http.
kovidgoyal is offline   Reply With Quote
Old 07-19-2011, 10:36 AM   #8
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
The sticky here explains that builtins are automatically updated when they are used and that Kovid does the updating. A few days ago I added to the sticky suggesting that people clearly identify recipes here that they think should replace existing builtins. Kovid keeps a close eye on posts here. He picks up new recipes and changes to existing ones to add to the repository. It's helpful for each author to make it clear in his post when he thinks the changes are good for everyone (and should replace the builtin) or if they are just for a special case or special device.
Starson17 is offline   Reply With Quote
Old 07-19-2011, 05:29 PM   #9
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
Thank you both for the participation and information.
khromov is offline   Reply With Quote
Old 07-24-2011, 06:34 PM   #10
ubieubie
Member
ubieubie began at the beginning.
 
Posts: 10
Karma: 10
Join Date: Dec 2010
Device: none
is there a limit to the number of articles that this recipe will send.
Thanks for the updated recipe
ubieubie is offline   Reply With Quote
Old 07-25-2011, 10:22 AM   #11
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by ubieubie View Post
is there a limit to the number of articles that this recipe will send.
Thanks for the updated recipe
Yes - a year old and no more than 100 articles. You can change those in the recipe, but it's doubtful those are really the limits. The true limit will probably be the number of articles the recipe finds, and that's set by the site.
Starson17 is offline   Reply With Quote
Old 07-25-2011, 12:25 PM   #12
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
I have updated the recipe to fix a problem with Instapaper where article titles are not being displayed inside the article body.

The update recipe is in the first post.
khromov is offline   Reply With Quote
Old 07-29-2011, 02:50 PM   #13
cendalc
Junior Member
cendalc began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Jul 2011
Device: Nook
Thanks for the script. I had several minor issues with it:

1. wrong czech characters - fixed by forcing utf8 and embeding own fonts. Of course the font path is hardcoded according to my Nook.
Code:
    encoding                = 'utf8'
    extra_css               = '''
@font-face {
    font-style: italic;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: normal;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Italic.ttf');
}
@font-face {
    font-style: normal;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: normal;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Regular.ttf');
}
@font-face {
    font-style: italic;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: bold;
    src:url('res:///system/media/sdcard/my fonts/LiberationSerif-BoldItalic.ttf');
}
@font-face {
    font-style: normal;
    font-family: 'LiberationSerif', serif, sans-serif;
    font-weight: bold;
    src: url('res:///system/media/sdcard/my fonts/LiberationSerif-Bold.ttf');
}
body {
    font-family: 'LiberationSerif', serif;
}'''
2. It does not archive downloaded articles. I updated parse_index method to retrieve form key and added cleanup method:

Code:
    def parse_index(self):
        totalfeeds = []
        lfeeds = self.get_feeds()
        for feedobj in lfeeds:
            feedtitle, feedurl = feedobj
            self.report_progress(0, _('Fetching feed')+' %s...'%(feedtitle if feedtitle else feedurl))
            articles = []
            soup = self.index_to_soup(feedurl)
            self.myFormKey = soup.find('input', attrs={'name': 'form_key'})['value']
            for item in soup.findAll('div', attrs={'class':'cornerControls'}):
                description = self.tag_to_string(item.div)
                atag = item.a
                if atag and atag.has_key('href'):
                    url         = atag['href']
                    articles.append({
                                     'url'        :url
                                    })
            totalfeeds.append((feedtitle, articles))
        return totalfeeds

    def cleanup(self):
        params = urllib.urlencode(dict(form_key=self.myFormKey, submit="Archive All"))
        self.browser.open("http://www.instapaper.com/bulk-archive", params)
Thanks to banjopicker

3. Multipart articles in wrong order - fixed by
Code:
    reverse_article_order = True
cendalc is offline   Reply With Quote
Old 07-31-2011, 07:43 AM   #14
khromov
Connoisseur
khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.khromov ought to be getting tired of karma fortunes by now.
 
Posts: 82
Karma: 499152
Join Date: Jul 2011
Device: Kindle
Hey cendalc! Thanks for the input on the recipe. I'd love to implement some of your fixes, but I need some more information.

1.) Can you provide for me the link to a czech article so I can see if Kindle is affected by the encoding issue? Hopefully adding utf-8 as the encoding will be enough for most devices.

2.) Not archiving downloaded articles is by design, since reading old papers on a Kindle is a clunky process. This way you can archive the articles you read on the Instapaper site and get all your unread ones the next day. I'll see about adding an option flag for it though so users can pick their preferred behaviour.

3.) Can you please provide the link to one (or more) multipart articles that I can test the incorrect order behaviour on?

Last edited by khromov; 07-31-2011 at 07:49 AM.
khromov is offline   Reply With Quote
Old 08-05-2011, 02:56 AM   #15
cendalc
Junior Member
cendalc began at the beginning.
 
Posts: 5
Karma: 10
Join Date: Jul 2011
Device: Nook
Hi Zapt0,

1, 3) Here is a link to czech multipart article: http://www.diit.cz/clanek/mobilni-op...android/38512/ The list of content at the begining is still wrong (there is for example Mobilní instead of Mobilní) but the content itself is ok.

As to multipart - I just went through all chapters (kapitola) and added them to Instapaper manually - that's why I set reverse order. Do you somehow try to recognize multipart articles and download them at once or correct their order?

I also set reverse order because I like reading my articles in the same order they were added.

2) My use case is I download articles automatically everyday so I don't want to download the same articles that were downloaded yesterday.
cendalc is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Nature news - updated recipe Alexis Recipes 3 10-05-2012 02:36 PM
Recipe for National Geographic *Updated* gagsays Recipes 3 05-19-2011 12:30 PM
Updated Recipe: They Draw and Cook Starson17 Recipes 0 03-13-2011 10:59 AM
Updated recipe for Le Monde? veezh Recipes 5 01-20-2011 09:06 PM
One new recipe and other one updated (In Spanish) desUBIKado Recipes 3 01-19-2011 03:58 AM


All times are GMT -4. The time now is 08:29 AM.


MobileRead.com is a privately owned, operated and funded community.