Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Closed Thread
 
Thread Tools Search this Thread
Old 07-04-2010, 11:04 AM   #2251
rty
Zealot
rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.
 
Posts: 105
Karma: 6066
Join Date: Apr 2010
Location: Travelling Nomad in Asia
Device: iPad 2, Nook 3G, Kindle DXG, Eken M001
Quote:
Originally Posted by Gunnerp245 View Post
Standard calibre news recipes:

English (Thailand) Bangkok Post.
English (Singapore) Today Online - Singapore.

Each only downloads the section headings; no articles.

I just went to check Today Online website RSS feed page http://www.todayonline.com/RSS/Hotnews and found that there is no feed on the page. Maybe they are updating the page now. We can check again later.

Recipe for Bangkok Post does not work anymore. If the author doesn't fix it, I'll probably can take a look at it this weekend.

Last edited by rty; 07-04-2010 at 11:17 AM.
rty is offline  
Old 07-04-2010, 11:12 AM   #2252
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,931
Karma: 5036099
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
http auth in mechanize is really easy there are examples here:

http://wwwsearch.sourceforge.net/mechanize/
kovidgoyal is offline  
Old 07-04-2010, 11:26 AM   #2253
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by kovidgoyal View Post
http auth in mechanize is really easy there are examples here:
http://wwwsearch.sourceforge.net/mechanize/
Thanks. The mechanize documentation seems particularly skimpy. I briefly read up on Google authentication, and it looked like they would occasionally issue a new CAPTCHA challenge after some failed login attempts. Not having an account, I wasn't sure how it all worked.
Starson17 is offline  
Old 07-04-2010, 07:22 PM   #2254
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,838
Karma: 12535517
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
Quote:
Originally Posted by Starson17 View Post
If someone wants to set up a test account, add some blogs or sites, or whatever content is needed in Google, put some "stars" or whatever they are into it, give me the user/pass (here or by PM) and help with figuring out what I'm supposed to be getting, I'll take a whack at it.
I'll setup a test account later tonight and send you what I know about its working.
DoctorOhh is offline  
Old 07-05-2010, 07:53 AM   #2255
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by dwanthny View Post
I'll setup a test account later tonight and send you what I know about its working.
OK, I've set up a test environment, added some debug statements into the old GoogleReader recipe, and drafted a first attempt at the new login/auth procedure. As soon as I have a valid user/pass, I'll run some tests.
Starson17 is offline  
Old 07-05-2010, 09:19 AM   #2256
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,838
Karma: 12535517
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
Quote:
Originally Posted by Starson17 View Post
OK, I've set up a test environment, added some debug statements into the old GoogleReader recipe, and drafted a first attempt at the new login/auth procedure. As soon as I have a valid user/pass, I'll run some tests.
The "star" thing described never worked. I always put whatever subscriptions I wanted to download in folders. The recipes treated each folder as a different feed and created a menu item for each.
DoctorOhh is offline  
Old 07-05-2010, 09:31 AM   #2257
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by Starson17 View Post
As to the "Photo" issue - you want to skip articles that have that text in the link. I only know one way to do that. Perhaps someone else knows another. Basically, I know two ways to follow articles - to follow all the links in the automatically parsed feed, or to build your own feed (without the Photo links) with parse_index and then follow all of those links.

If there's another way - to follow some links, but not others, I don't know it.
I thought of "another way."

Use print_version.
Try this code:
Code:
    def print_version(self, url):
        match = re.search(r'PhotoGallery', url)
        if not match:
            return url
Print_version operates between the parsing and the fetching. The code above will check to see if "PhotoGallery" is in the URL and skip that article if it is.
Starson17 is offline  
Old 07-05-2010, 09:35 AM   #2258
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by dwanthny View Post
The "star" thing described never worked. I always put whatever subscriptions I wanted to download in folders. The recipes treated each folder as a different feed and created a menu item for each.
I assume this was for the builtin GoogleReader recipe - correct? There's also something called GoogleReader Uber. Is that involved in this?
Starson17 is offline  
Old 07-05-2010, 01:04 PM   #2259
rty
Zealot
rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.
 
Posts: 105
Karma: 6066
Join Date: Apr 2010
Location: Travelling Nomad in Asia
Device: iPad 2, Nook 3G, Kindle DXG, Eken M001
My humble little gift to folks up there in Alaska: Recipe for Anchorage Daily News.
Attached Files
File Type: zip Anchorage Daily News.zip (770 Bytes, 62 views)
rty is offline  
Old 07-05-2010, 09:09 PM   #2260
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 8,838
Karma: 12535517
Join Date: Feb 2009
Location: North Carolina
Device: Nexus 7
Quote:
Originally Posted by Starson17 View Post
I assume this was for the builtin GoogleReader recipe - correct? There's also something called GoogleReader Uber. Is that involved in this?
That is correct. GoogleReader is the one I have been referring to. To save my brain I actually posted the difference between them a while back when I understood the difference.

Although the changes you make for Google Reader should work for Google reader Uber too. they both use the SID for login.

Last edited by DoctorOhh; 07-05-2010 at 09:16 PM.
DoctorOhh is offline  
Old 07-05-2010, 09:58 PM   #2261
mik991
Junior Member
mik991 began at the beginning.
 
Posts: 4
Karma: 10
Join Date: May 2010
Device: Nook
Local Newspaper

Hi Guys I was trying to create a custom recipe for the local newspaper, Democrat and Chronicle. I found the print page and that works very fast with calibre. My problem is once it syncs onto my nook it will only display one page worth for each story and then cut off (sometimes even the bottom portion of the next sentence). I'm sure it is most likely some code snippet I am missing.

Any help would be greatly appreciated.

Here is what I had
Code:
class AdvancedUserRecipe1278352006(BasicNewsRecipe):
    title          = u'Democrat and Chronicle'
    oldest_article = 1
    max_articles_per_feed = 100
    no_stylesheets = True

    feeds          = [(u'Breaking news and top stories', u'http://www.democratandchronicle.com/apps/pbcs.dll/section?category=RSS&mime=xml'), 

(u'News', u'http://www.democratandchronicle.com/apps/pbcs.dll/section?category=RSS01&mime=xml'), 

(u'Bills', u'http://www.democratandchronicle.com/apps/pbcs.dll/section?category=SPORTS03&template=rss_dc&mime=xml'), 

(u'Business', u'http://www.democratandchronicle.com/apps/pbcs.dll/section?category=BUSINESS&template=rss_dc&mime=xml'), 

(u'Health', u'http://www.democratandchronicle.com/apps/pbcs.dll/section?category=HEALTH&template=rss_dc&mime=xml'), 

(u'Arts', u'http://www.democratandchronicle.com/apps/pbcs.dll/section?category=rssdcarts&mime=xml'), 

(u'Living', u'http://www.democratandchronicle.com/apps/pbcs.dll/section?category=LIVING&template=rss_dc&mime=xml'), 

(u'Opinions', u'http://www.democratandchronicle.com/apps/pbcs.dll/section?category=RSS04&mime=xml'), 

(u'Sports', u'http://www.democratandchronicle.com/apps/pbcs.dll/section?category=SPORTS&template=rss_dc&mime=xml')]

    def print_version(self, url):
    	return url.replace('http://www.democratandchronicle.com/', 'http://www.democratandchronicle.com/print/')
mik991 is offline  
Old 07-06-2010, 05:29 AM   #2262
mikegps1
Junior Member
mikegps1 began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Jul 2010
Device: sony prs600
Request for Timesonline and Sunday Times (UK)

The UK online editions of The Times and The Sunday Times are now subscription only. I have a subscription for both and would like to download on to my Sony PRS600. The existing custom recipe doesn't have the facility for sending the password - can any of the wizards on this forum help please?

The urls are:
http://www.thetimes.co.uk/tto/news/
http://www.thesundaytimes.co.uk/sto/

The Calibre program is excellent - if Sony had any sense they should be throwing large sums of money at Kovid to allow them to ship Calibre with their products instead of the inferior Reader program.

Thanks in advance.

Mike
mikegps1 is offline  
Old 07-06-2010, 02:49 PM   #2263
nook.life
Member
nook.life began at the beginning.
 
Posts: 12
Karma: 10
Join Date: May 2010
Device: Nook
Working Cyanide and Happiness Recipe (Explosm)

Quote:
Originally Posted by Starson17 View Post
There is a <div> tag in the original that has a style with a defined width that matches the width of the image before rotation, and a text-align attribute that centers the image in that width.

I don't have a nook, but try this...
Starson17, you are the best. The recipe works perfectly now. It is centered and everything. Thank you so, so much for all of the time you put into this. For those of you who want the final recipe here it is. Be sure to thank Starson17 for all of his hard work. Thanks again!

Spoiler:
Code:
from calibre.web.feeds.news import BasicNewsRecipe
from calibre.ebooks.BeautifulSoup import BeautifulSoup
import re
import calibre.utils.PythonMagickWand as pw
import calibre.utils.PythonMagickWand

class Explosm(BasicNewsRecipe):
    title               = 'Explosm Rotated'
    __author__          = 'Starson17'
    description         = 'Explosm'
    language            = 'en'
    use_embedded_content= False
    no_stylesheets      = True
    oldest_article      = 24
    remove_javascript   = True
    remove_empty_feeds    = True
    max_articles_per_feed = 10

    feeds = [
             (u'Explosm Feed', u'http://feeds.feedburner.com/Explosm')
             ]

    keep_only_tags     = [dict(name='div', attrs={'align':'center'})]
    remove_tags = [dict(name='span'),
                   dict(name='table')]

    def postprocess_html(self, soup, first):
        #process all the images. assumes that the new html has the correct path
        for tag in soup.findAll(lambda tag: tag.name.lower()=='img' and tag.has_key('src')):
            iurl = tag['src']
            print 'resizing image' + iurl
            with pw.ImageMagick():
                img = pw.NewMagickWand()
                p = pw.NewPixelWand()
                if img < 0:
                    raise RuntimeError('Out of memory')
                if not pw.MagickReadImage(img, iurl):
                    severity = pw.ExceptionType(0)
                    msg = pw.MagickGetException(img, byref(severity))
                    raise IOError('Failed to read image from: %s: %s'
                        %(iurl, msg))
                width = pw.MagickGetImageWidth(img)
                height = pw.MagickGetImageHeight(img)
                if( width > height ) :
                    print 'Rotate image'
                    pw.MagickRotateImage(img, p, 90)
                if not pw.MagickWriteImage(img, iurl):
                    raise RuntimeError('Failed to save image to %s'%iurl)
                pw.DestroyMagickWand(img)
                for divtag in soup.findAll('div'):
                   del(divtag['style'])
        return soup

    extra_css = '''
                    h1{font-family:Arial,Helvetica,sans-serif; font-weight:bold;font-size:large;}
                    h2{font-family:Arial,Helvetica,sans-serif; font-weight:normal;font-size:small;}
                    p{font-family:Arial,Helvetica,sans-serif;font-size:small;}
                    body{font-family:Helvetica,Arial,sans-serif;font-size:small;}
		'''
nook.life is offline  
Old 07-06-2010, 02:50 PM   #2264
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by mikegps1 View Post
The Calibre program is excellent - if Sony had any sense they should be throwing large sums of money at Kovid to allow them to ship Calibre with their products instead of the inferior Reader program.
Calibre is licensed to all under GPL v.3. That means Sony wouldn't have to pay a dime to Kovid (or me or anyone else who has contributed to the Calibre code). OTOH, they would have to comply with the terms of GPL v.3.
Starson17 is offline  
Old 07-06-2010, 03:12 PM   #2265
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by nook.life View Post
Starson17, you are the best. The recipe works perfectly now. It is centered and everything. Thank you so, so much for all of the time you put into this.
I'm glad it's working for you. I realized after I wrote that little bit of code that there's a predefined method that I had never used that would have done it for you more easily:

Code:
remove_attributes = ['style']
As an aside, I often wish I could have the recipe accept some settings from the user, other than just the username and password. In this recipe, the images are rotated, which makes sense for someone reading on a handheld device, but not for someone who reads on a desktop. It would be nice to simply have an option to rotate images or not, instead of requiring two different recipes that are basically the same.

Most recipes have optional settings, such as the number of comics to download, the number of articles to retrieve, etc. It would be nice to have a settings screen associated with the recipe (like the user/password) that can be set by the user when they schedule a builtin. The way it works now, the user has to load the builtin recipe in the recipe editor, modify the code, save the recipe, go to the News fetcher and schedule the user modified version.
Starson17 is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Custom column read ? pchrist7 Calibre 2 10-04-2010 02:52 AM
Archive for custom screensavers sleeplessdave Amazon Kindle 1 07-07-2010 12:33 PM
How to back up preferences and custom recipes? greenapple Calibre 3 03-29-2010 05:08 AM
Donations for Custom Recipes ddavtian Calibre 5 01-23-2010 04:54 PM
Help understanding custom recipes andersent Calibre 0 12-17-2009 02:37 PM


All times are GMT -4. The time now is 04:44 PM.


MobileRead.com is a privately owned, operated and funded community.