Request for Recipes: Yakima Valley Newspapers

[Tegan]

Tri-City Herald Mid-Columbia - http://www.tri-cityherald.com/901/index.rss
Yakima Herald Online - http://feeds.feedburner.com/yhronlinenews


I'd also like to see a recipe for the Seattle PI, which is online only now:
Local News - http://www.seattlepi.com/rss/local_2.rss
Opinion - http://www.seattlepi.com/rss/opinion_2.rss

Other Seattle PI feeds are listed at:
http://www.seattlepi.com/rss/


Also, is there a way to rotate the images in the gocomics or comics feeds to make 'em display larger (and therefore better) in my Kindle? Ideally, I would love a way to have the images in a comics recipe display the shortest side as the width and the longest side as the height, rotating them only if it makes sense. I'm guessing that would be a really hard fix requiring coding change, though, so I thought I'd just ask if it already exists or is likely to ever exist.

[Tegan]

Ok, I've done my own recipe for the first two links. I put them together, because they are both of interest to me here. My solutions weren't particularly elegant, I just kept trying to figure out how to snip away the stuff that wasn't of interest. In the Tri-City Herald I lost all the images because I couldn't figure out how to display them without a whole bunch of extra text.

I think I'll take a crack at the Seattle PI feeds now, but there are a few other sites I want to try now that I have a slight notion of what I can do.

[Tegan]

Ok, I've got an odd one... I redid the Tri-City Herald Mid-Columbia recipe to try to recover those images. And it worked, I got 'em. But they show up at the bottom of the article instead of under the headline with the caption. Is there a way to move the items within an article around, so I could either move the image up or put the caption down where it belongs?

class AdvancedUserRecipe1294886492(BasicNewsRecipe):
    title          = u'Tri-City Herald'
    description    = 'The Tri-City Herald Mid-Columbia.'
    __author__     = 'Laura Gjovaag'
    oldest_article        = 1.5
    max_articles_per_feed = 100
    no_stylesheets        = True
    remove_javascript     = True
    keep_only_tags = [
                      dict(name='div', attrs={'id':['story_header', 'cycleImageCaption', 'story_body']})
                      dict(name='img', attrs={'class':'imageCycle'}),
                      ]
    remove_tags    = [
                      dict(name='div', attrs={'id':'story_mlt'}),
                      dict(name='a', attrs={'id':'commentCount'}),
                      dict(name=['script', 'noscript', 'style'])]
    extra_css      = 'h1{font: bold 140%;} #cycleImageCaption{font: monospace 60%}'

    feeds          = [
                      (u'Tri-City Herald Mid-Columbia', u'http://www.tri-cityherald.com/901/index.rss')
                     ]

[kovidgoyal]

The postprocess_html method is what you're looking for.

[Tegan]

Ok, fixed it. Amazing what the placement of a couple of lines can do...

class TriCityHeraldRecipe(BasicNewsRecipe):
    title          = u'Tri-City Herald'
    description    = 'The Tri-City Herald Mid-Columbia.'
    __author__     = 'Laura Gjovaag'
    oldest_article        = 1.5
    max_articles_per_feed = 100
    no_stylesheets        = True
    remove_javascript     = True
    keep_only_tags = [
                      dict(name='div', attrs={'id':'story_header'}),
                      dict(name='img', attrs={'class':'imageCycle'}),
                      dict(name='div', attrs={'id':['cycleImageCaption', 'story_body']})
                      ]
    remove_tags    = [
                      dict(name='div', attrs={'id':'story_mlt'}),
                      dict(name='a', attrs={'id':'commentCount'}),
                      dict(name=['script', 'noscript', 'style'])]
    extra_css      = 'h1{font: bold 140%;} #cycleImageCaption{font: monospace 60%}'

    feeds          = [
                      (u'Tri-City Herald Mid-Columbia', u'http://www.tri-cityherald.com/901/index.rss')
                     ]

And here's the one for the Yakima Herald-Republic...

class YakimaHeraldRepublicRecipe(BasicNewsRecipe):
    title          = u'Yakima Herald-Republic'
    description    = 'The Yakima Herald-Republic.'
    __author__     = 'Laura Gjovaag'
    oldest_article        = 1.5
    max_articles_per_feed = 100
    no_stylesheets        = True
    remove_javascript     = True
    keep_only_tags = [
                      dict(name='div', attrs={'id':['searchleft', 'headline_credit']}),
                      dict(name='div', attrs={'class':['photo', 'cauthor', 'photocredit']}),
                      dict(name='div', attrs={'id':['content_body', 'footerleft']})
                      ]
    extra_css = '.cauthor {font: monospace 60%;} .photocredit {font: monospace 60%}'

    feeds          = [
                      (u'Yakima Herald Online', u'http://feeds.feedburner.com/yhronlinenews'), 
                     ]

[Tegan]

I've found some code that is used to rotate images just like I described, over in this thread: https://www.mobileread.com/forums/sho...postcount=2263

But can it be applied to gocomics or comics.com recipes? I honestly can't parse it. Can anyone explain the recipe in that post to me?

[Starson17]

Originally Posted by Tegan

I've found some code that is used to rotate images just like I described, over in this thread: https://www.mobileread.com/forums/sho...postcount=2263

But can it be applied to gocomics or comics.com recipes? I honestly can't parse it. Can anyone explain the recipe in that post to me?

I wrote those two comic recipes. If your question is about them, feel free to ask. The link you've posted goes to some older code that I'm familiar with, but which used PythonMagickWand to do the image manipulation. It won't work anymore, as written, but worked fine originally in all comics and image-based recipes. You'll have to ask Kovid exactly what happened to PythonMagickWand and what can substitute for it.

I just rotate my reader from portrait to landscape and the G-sensor redoes the image. The CSS for images in those recipes is set to make the width equal to the max width of the device.

[kovidgoyal]

python magick wand was replaced by calibre.utils.magick and calibre.utils.magick.draw

[Tegan]

Originally Posted by Starson17

I just rotate my reader from portrait to landscape and the G-sensor redoes the image. The CSS for images in those recipes is set to make the width equal to the max width of the device.

Yeah, I've got a Kindle 1, so rotation of the reader itself doesn't do anything.

I just am trying to learn the code for the recipes by taking apart old code and trying to figure it out, while in the meantime making the recipes work better for me. I understand the bit about rotation, but I still haven't quite worked out what the lambda call does.

My understanding of that code is that it calls the graphic editor (Magick Wand) which Kovid points out has been replaced with a new call. Ok, got that. Then it does the postprocess to rotate the image. I understand the gist of it, but I'm an English grad, not a programmer, so I'm not sure what each bit is doing and searches aren't yielding much help.

Line by line:

for tag in soup.findAll(lambda tag: tag.name.lower()=='img' and tag.has_key('src')):

This is the bit that confuses me most. Is there a beginners guide to soup? What is the lambda? If someone can help me with that, maybe I can figure it out for myself.

 with pw.ImageMagick():
                img = pw.NewMagickWand()
                p = pw.NewPixelWand()

Ok, this is calling the image editor? How would that change with the new names? Where does one find the names of the Magick calls?

width = pw.MagickGetImageWidth(img)
                height = pw.MagickGetImageHeight(img)
                if( width > height ) :
                    print 'Rotate image'
                    pw.MagickRotateImage(img, p, 90)
                if not pw.MagickWriteImage(img, iurl):
                    raise RuntimeError('Failed to save image to %s'%iurl)
                pw.DestroyMagickWand(img)
                for divtag in soup.findAll('div'):
                   del(divtag['style'])
        return soup

Same here. I see a bunch of different calls to the image editor, but I can't find any documentation on how those calls should be phrased in the recipe. I've been Googling like mad, and still can't figure out some of the basics. I think there is a level of programming that I'm missing.

Can this bit of code be rewritten to apply to the current GoComics and Comics.com recipes so they will be automagically rotated for me? I'd be delighted if so. It would keep me from having to squint at the comics every day.

[Starson17]

Originally Posted by Tegan

I still haven't quite worked out what the lambda call does.

Python supports the creation of anonymous functions (i.e. functions that are not bound to a name), using the "lambda" construct.

Line by line:

for tag in soup.findAll(lambda tag: tag.name.lower()=='img' and tag.has_key('src')):

This is the bit that confuses me most.

Simple - it finds img tags (convert to lower case first and compare tag name to "img") that have a key of "src."

Is there a beginners guide to soup?

http://www.crummy.com/software/Beaut...mentation.html

How would that change with the new names? Where does one find the names of the Magick calls?

We'll have to let Kovid answer that or point us to documentation.

I see a bunch of different calls to the image editor, but I can't find any documentation on how those calls should be phrased in the recipe. I've been Googling like mad, and still can't figure out some of the basics. I think there is a level of programming that I'm missing.

All that's needed are the Magick calls that read the img, tell us the height and width for the compare, rotate the image if needed, and write it back.

Can this bit of code be rewritten to apply to the current GoComics and Comics.com recipes so they will be automagically rotated for me?

Yes, provided that the image rotate function is available. I haven't had a chance to go Googling on this.

[Tegan]

Originally Posted by Starson17

Python supports the creation of anonymous functions (i.e. functions that are not bound to a name), using the "lambda" construct.

Googling it. I was thinking it was something entirely different, and now you've got me on the right path. Thank you!

Originally Posted by Starson17

http://www.crummy.com/software/Beaut...mentation.html

This should help. Again, thank you!

Originally Posted by Starson17

We'll have to let Kovid answer that or point us to documentation.

Something like this: http://calibre.sourcearchive.com/doc...4190f4fa7.html
Or is that just the source code? I need documentation, but I don't want to pull Kovid away from improving his program to write up stuff to help people like me.

In any case, thank you for answering. You've set me on the right path to figuring it out.

[kovidgoyal]

Just look at the file comic/input.py for an example of using the new ImageMagick interface. It includes code to rotate/resize images.

[Starson17]

Thanks to Kovid.
I rewrote the image rotation code. It's here in the sticky reusable code thread:

https://www.mobileread.com/forums/sho...7&postcount=15

If you use it in my comics recipes, remember that you will need to control image sizing with extra CSS. The current recipes set image width to 100%. You may wish to change it to set image height to 100% as this code will always have the longer image dimension set vertically.

[Tegan]

Originally Posted by Starson17

Thanks to Kovid.
I rewrote the image rotation code.

Can't seem to get it to work. I always end up with no images in the final product. Once I have a little time I'll try to troubleshoot it.

[Starson17]

Originally Posted by Tegan

Can't seem to get it to work. I always end up with no images in the final product. Once I have a little time I'll try to troubleshoot it.

Post your recipe if you need help. It works for me. I tested in the Explosm recipe (at your original link) for which that code was originally designed, but it should work in others.