Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Closed Thread
 
Thread Tools Search this Thread
Old 06-27-2010, 02:07 AM   #2206
plaid
Connoisseur
plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!
 
Posts: 94
Karma: 5434
Join Date: Sep 2004
Device: Kobo eReader
Winnipeg Sun

Is it possible for Calibre to pull down a mobile site rather than rss feeds? For local Winnipeg Canada news all the rss feeds I can find have only one line of text, hardly worth reading. The Winnipeg Sun has a mobile version that will let me see the whole articles. I did some experimenting and looked at the recipe guides but I've not had any luck. The ultimate destination of the news is my Kobo

http://m.winnipegsun.com/
plaid is offline  
Old 06-27-2010, 05:55 AM   #2207
rty
Zealot
rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.
 
Posts: 105
Karma: 6066
Join Date: Apr 2010
Location: Travelling Nomad in Asia
Device: iPad 2, Nook 3G, Kindle DXG, Eken M001
Quote:
Originally Posted by plaid View Post
Is it possible for Calibre to pull down a mobile site rather than rss feeds? For local Winnipeg Canada news all the rss feeds I can find have only one line of text, hardly worth reading. The Winnipeg Sun has a mobile version that will let me see the whole articles. I did some experimenting and looked at the recipe guides but I've not had any luck. The ultimate destination of the news is my Kobo

http://m.winnipegsun.com/
I took a glance at it and it looks OK. Give me a week.
rty is offline  
Old 06-27-2010, 11:16 AM   #2208
plaid
Connoisseur
plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!
 
Posts: 94
Karma: 5434
Join Date: Sep 2004
Device: Kobo eReader
Quote:
Originally Posted by rty View Post
I took a glance at it and it looks OK. Give me a week.
Ok, that would be great. I'll try to keep an eye on this thread, thanks!
plaid is offline  
Old 06-27-2010, 11:57 AM   #2209
rty
Zealot
rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.
 
Posts: 105
Karma: 6066
Join Date: Apr 2010
Location: Travelling Nomad in Asia
Device: iPad 2, Nook 3G, Kindle DXG, Eken M001
To all the good folks in Winnipeg, here is the recipe for Winnipeg Sun.


@plaid: The RSS feeds are used only to pull down the full artilces.
Attached Files
File Type: zip WinnipegSun.zip (652 Bytes, 66 views)
rty is offline  
Old 06-27-2010, 04:49 PM   #2210
plaid
Connoisseur
plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!plaid , Klaatu Barada Niktu!
 
Posts: 94
Karma: 5434
Join Date: Sep 2004
Device: Kobo eReader
Quote:
Originally Posted by rty View Post
To all the good folks in Winnipeg, here is the recipe for Winnipeg Sun.


@plaid: The RSS feeds are used only to pull down the full artilces.
THANKS! it works great.
plaid is offline  
Old 06-28-2010, 06:01 PM   #2211
nook.life
Member
nook.life began at the beginning.
 
Posts: 12
Karma: 10
Join Date: May 2010
Device: Nook
Quote:
Originally Posted by Starson17 View Post
I was probably in a grumpy mood that day

Whatever I posted, it wasn't the final recipe, as what you were working with still had lots of junk in it. This is closer to the final I came up with, but my earlier version had some text that identified the comic. You want it rotated, so I removed the text above to give more room for the comic.

Try this:
Spoiler:
Code:
from calibre.web.feeds.news import BasicNewsRecipe
from calibre.ebooks.BeautifulSoup import BeautifulSoup
import re
import calibre.utils.PythonMagickWand as pw
import calibre.utils.PythonMagickWand

class Explosm(BasicNewsRecipe):
    title               = 'Explosm'
    __author__          = 'Starson17'
    description         = 'Explosm'
    language            = 'en'
    use_embedded_content= False
    no_stylesheets      = True
    oldest_article      = 24
    remove_javascript   = True
    remove_empty_feeds    = True
    max_articles_per_feed = 10

    feeds = [
             (u'Explosm Feed', u'http://feeds.feedburner.com/Explosm')
             ]

    keep_only_tags     = [dict(name='div', attrs={'align':'center'})]
    remove_tags = [dict(name='span'),
                   dict(name='table')]

    def postprocess_html(self, soup, first):
        #process all the images. assumes that the new html has the correct path
        for tag in soup.findAll(lambda tag: tag.name.lower()=='img' and tag.has_key('src')):
            iurl = tag['src']
            print 'resizing image' + iurl
            with pw.ImageMagick():
                img = pw.NewMagickWand()
                p = pw.NewPixelWand()
                if img < 0:
                    raise RuntimeError('Out of memory')
                if not pw.MagickReadImage(img, iurl):
                    severity = pw.ExceptionType(0)
                    msg = pw.MagickGetException(img, byref(severity))
                    raise IOError('Failed to read image from: %s: %s'
                        %(iurl, msg))
                width = pw.MagickGetImageWidth(img)
                height = pw.MagickGetImageHeight(img)
                if( width > height ) :
                    print 'Rotate image'
                    pw.MagickRotateImage(img, p, 90)
                if not pw.MagickWriteImage(img, iurl):
                    raise RuntimeError('Failed to save image to %s'%iurl)
                pw.DestroyMagickWand(img)
        return soup

    extra_css = '''
                    h1{font-family:Arial,Helvetica,sans-serif; font-weight:bold;font-size:large;}
                    h2{font-family:Arial,Helvetica,sans-serif; font-weight:normal;font-size:small;}
                    p{font-family:Arial,Helvetica,sans-serif;font-size:small;}
                    body{font-family:Helvetica,Arial,sans-serif;font-size:small;}
		'''
Hey Starson, thanks so much for posting the code. It definitely looks a lot cleaner without the text above and makes it more readable. Unfortunately, it is still clipping it at the top for some of the comics. I looked at the html test version and it still seems to be outputting some sort of table (you can see the outline) even though you removed it. Starting to wonder if my output settings are messed up in calibre. I uninstalled and reinstalled it, but it seems like it keeps the settings (my custom recipes and scheduled were still there after the new install)

here is what it looks like
http://picturepush.com/public/3708883
nook.life is offline  
Old 06-28-2010, 08:13 PM   #2212
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by nook.life View Post
Unfortunately, it is still clipping it at the top for some of the comics.
That seems to be the right, not the top. I don't have a Nook, so can't do much testing.

Quote:
I looked at the html test version and it still seems to be outputting some sort of table (you can see the outline) even though you removed it.
I checked my output - there aren't any tables there. Did you see a table tag in your output?
Starson17 is offline  
Old 06-28-2010, 08:36 PM   #2213
nook.life
Member
nook.life began at the beginning.
 
Posts: 12
Karma: 10
Join Date: May 2010
Device: Nook
Quote:
Originally Posted by Starson17 View Post
That seems to be the right, not the top. I don't have a Nook, so can't do much testing.


I checked my output - there aren't any tables there. Did you see a table tag in your output?
My apologies, I was referring to the top of the cartoon (since it is rotated).

No there are no table tags since you already took it out with the code, but for some reason it is still doing some sort of formatting where it draws a frame and places the image inside it. When I open the html in mozialla you can see the frame.

In the epub viewer within calibre it shows the cartoon clipped on the right like you say, but because it has a scroll bar at the bottom you can scroll and see the rest of the cartoon. The nook does not have such scroll bar so it crops the image as shown by the pic i posted (and deletes the rest?). I just dont know how to get rid of that extra space on the left.

I do not experience this problem with other cartoon feeds like Dilbert or comics.com even though I pasted in the rotate code to rotate them.

Last edited by nook.life; 06-28-2010 at 08:38 PM.
nook.life is offline  
Old 06-29-2010, 05:24 AM   #2214
Dereks
Connoisseur
Dereks began at the beginning.
 
Posts: 55
Karma: 10
Join Date: Feb 2010
Device: Sony PRS-T1
For some reason i can no longer download google reader recipe. Starting from 0.7.x version I get this error:

Quote:
ERROR: Conversion Error: <b>Failed</b>: Fetch news from Google Reader

Fetch news from Google Reader
Resolved conversion options
calibre version: 0.7.6
{'asciiize': False,
'author_sort': None,
'authors': None,
'base_font_size': 0,
'book_producer': None,
'change_justification': 'original',
'chapter': None,
'chapter_mark': 'pagebreak',
'comments': None,
'cover': None,
'debug_pipeline': None,
'disable_font_rescaling': False,
'dont_download_recipe': False,
'enable_autorotation': False,
'extra_css': None,
'font_size_mapping': None,
'footer_regex': '(?i)(?<=<hr>)((\\s*<a name=\\d+></a>((<img.+?>)*<br>\\s*)?\\d+<br>\\s*.*?\\s*)|(\\s* <a name=\\d+></a>((<img.+?>)*<br>\\s*)?.*?<br>\\s*\\d+))(?=<br>)' ,
'header': False,
'header_format': '%t by %a',
'header_regex': '(?i)(?<=<hr>)((\\s*<a name=\\d+></a>((<img.+?>)*<br>\\s*)?\\d+<br>\\s*.*?\\s*)|(\\s* <a name=\\d+></a>((<img.+?>)*<br>\\s*)?.*?<br>\\s*\\d+))(?=<br>)' ,
'header_separation': 0,
'input_encoding': None,
'input_profile': <calibre.customize.profiles.InputProfile object at 0x05189A50>,
'insert_blank_line': False,
'insert_metadata': False,
'isbn': None,
'keep_ligatures': False,
'language': None,
'level1_toc': None,
'level2_toc': None,
'level3_toc': None,
'line_height': 0,
'linearize_tables': False,
'lrf': False,
'margin_bottom': 5.0,
'margin_left': 5.0,
'margin_right': 5.0,
'margin_top': 5.0,
'max_toc_links': 50,
'minimum_indent': 0,
'mono_family': None,
'no_chapters_in_toc': False,
'no_inline_navbars': False,
'output_profile': <calibre.customize.profiles.SonyReaderOutput object at 0x05189DF0>,
'page_breaks_before': None,
'password': 'xxxxxxxxxxx',
'prefer_metadata_cover': False,
'preprocess_html': False,
'pretty_print': False,
'pubdate': None,
'publisher': None,
'rating': None,
'read_metadata_from_opf': None,
'remove_first_image': False,
'remove_footer': False,
'remove_header': False,
'remove_paragraph_spacing': False,
'remove_paragraph_spacing_indent_size': 1.5,
'render_tables_as_images': False,
'sans_family': None,
'series': None,
'series_index': None,
'serif_family': None,
'tags': None,
'test': False,
'text_size_multiplier_for_rendered_tables': 1.0,
'timestamp': None,
'title': None,
'title_sort': None,
'toc_filter': None,
'toc_threshold': 6,
'use_auto_toc': False,
'username': 'dereksts',
'verbose': 2,
'wordspace': 2.5}
InputFormatPlugin: Recipe Input running
Python function terminated unexpectedly
(Error Code: 1)
Traceback (most recent call last):
File "site.py", line 103, in main
File "site.py", line 85, in run_entry_point
File "site-packages\calibre\utils\ipc\worker.py", line 99, in main
File "site-packages\calibre\gui2\convert\gui_conversion.py", line 24, in gui_convert
File "site-packages\calibre\ebooks\conversion\plumber.py", line 815, in run
File "site-packages\calibre\customize\conversion.py", line 211, in __call__
File "site-packages\calibre\web\feeds\input.py", line 104, in convert
File "site-packages\calibre\web\feeds\news.py", line 705, in download
File "site-packages\calibre\web\feeds\news.py", line 835, in build_index
File "site-packages\calibre\web\feeds\news.py", line 1280, in parse_feeds
File "c:\users\dereks\appdata\local\temp\calibre_0.7.6_ 1vhut0_recipes\recipe0.py", line 35, in get_feeds
soup = self.index_to_soup('http://www.google.com/reader/api/0/tag/list')
File "site-packages\calibre\web\feeds\news.py", line 474, in index_to_soup
File "site-packages\mechanize-0.1.11-py2.6.egg\mechanize\_opener.py", line 202, in open
File "site-packages\mechanize-0.1.11-py2.6.egg\mechanize\_http.py", line 612, in http_response
File "site-packages\mechanize-0.1.11-py2.6.egg\mechanize\_opener.py", line 225, in error
File "urllib2.py", line 367, in _call_chain
File "site-packages\mechanize-0.1.11-py2.6.egg\mechanize\_http.py", line 633, in http_error_default
urllib2.HTTPError: HTTP Error 401: Unauthorized

Last edited by Dereks; 06-30-2010 at 06:25 AM.
Dereks is offline  
Old 07-01-2010, 11:20 AM   #2215
ehsahog
Junior Member
ehsahog began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Jul 2010
Device: Kindle DX, Bebook
Recipe request: Akihabara news

Hi all,

I just recently discovered Calibre and the feature to get news.
Unfortunately, I'm not very good at Phyton or HTML/CSS so I haven't been successful in my attempts to customize the recipes to get the news I want.

Would it be possible for someone with knowledge to write a recipe for akihabara news (http://en.akihabaranews.com/feed)

Lots of thanks in advance!
/Anders
ehsahog is offline  
Old 07-01-2010, 01:29 PM   #2216
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by nook.life View Post
No there are no table tags since you already took it out with the code, but for some reason it is still doing some sort of formatting where it draws a frame and places the image inside it. When I open the html in mozialla you can see the frame.... I just dont know how to get rid of that extra space on the left.

I do not experience this problem with other cartoon feeds like Dilbert or comics.com even though I pasted in the rotate code to rotate them.
There is a <div> tag in the original that has a style with a defined width that matches the width of the image before rotation, and a text-align attribute that centers the image in that width.

I don't have a nook, but try this: Paste this into the recipe after the rotate code (after pw.DestroyMagickWand(img)), and inside the postprocess_html code:

Code:
                for divtag in soup.findAll('div'):
                   del(divtag['style'])
This deletes the style and should let it put the image to the left.
Starson17 is offline  
Old 07-01-2010, 03:13 PM   #2217
sibermage
Junior Member
sibermage began at the beginning.
 
Posts: 3
Karma: 10
Join Date: Jul 2010
Device: Sony PRS600
Question Sing Tao News

I did a search but couldn't find anything regarding the Sing Tao news site.
Is it possible to obtain the news articles for http://news.singtao.ca/toronto/

Thanks.
sibermage is offline  
Old 07-01-2010, 08:02 PM   #2218
elsuave
Member
elsuave began at the beginning.
 
Posts: 12
Karma: 10
Join Date: Jun 2010
Device: Nook
Foreign Policy

Did a recipe for Foreign Policy (http://www.foreignpolicy.com/) ever come out? It's been mentioned a couple of times in this thread, with an unsuccessful attempt here: http://www.mobileread.com/forums/sho...&postcount=616.

If not, would anybody like to try their hand at it? RSS feed is available here: http://www.foreignpolicy.com/node/feed
elsuave is offline  
Old 07-01-2010, 10:36 PM   #2219
Daffy6964
Junior Member
Daffy6964 began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Jan 2010
Location: Texas
Device: Sony PRS-300
Austin Statesman

A recipe request for The Statesman. News for Austin, TX.
http://www.statesman.com/

Thank you!
Daffy6964 is offline  
Old 07-02-2010, 05:02 AM   #2220
rty
Zealot
rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.rty got an A in P-Chem.
 
Posts: 105
Karma: 6066
Join Date: Apr 2010
Location: Travelling Nomad in Asia
Device: iPad 2, Nook 3G, Kindle DXG, Eken M001
Recipe for the STATESMAN for folks in Austin, Texas!

Note: Articles in the feeds linked from blogs and Austin360.com are ignored.
Attached Files
File Type: zip Statesman.zip (709 Bytes, 68 views)
rty is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Custom column read ? pchrist7 Calibre 2 10-04-2010 03:52 AM
Archive for custom screensavers sleeplessdave Amazon Kindle 1 07-07-2010 01:33 PM
How to back up preferences and custom recipes? greenapple Calibre 3 03-29-2010 06:08 AM
Donations for Custom Recipes ddavtian Calibre 5 01-23-2010 05:54 PM
Help understanding custom recipes andersent Calibre 0 12-17-2009 03:37 PM


All times are GMT -4. The time now is 09:01 PM.


MobileRead.com is a privately owned, operated and funded community.