Custom recipes (archive, read-only)

[kovidgoyal]

No you should just need to remove the feeds you dont want

[kiklop74]

Originally Posted by hackettt

Kovid and Darko —

Is there something I must do besides eliminating the code for other sections I do not want?

You are complicating things without real need. This is what you need to change in your recipe:

class AdvancedUserRecipe1249153260(BasicNewsRecipe):
    title          = u'DailyMail'
    oldest_article = 2
    max_articles_per_feed = 100
    no_stylesheets = True
    encoding = 'cp1252'

    keep_only_tags = [dict(name='div', attrs={'id':'js-article-text'})]

    remove_tags = [dict(name='div', attrs={'class':['relatedItems','article-icon-links-container']})]
    
    remove_tags_after = dict(name='h3', attrs={'class':'social-links-title'})
    
    feeds          = [(u'Sports', u'http://www.dailymail.co.uk/sport/index.rss')]

    def print_version(self, url):
        main = url.partition('?')[0]
        return main + '?printingPage=true'

[fogus]

I did a search but I didn't find anything for the following idea:

I read a lot of fixed width (80 character often) texts. Does anyone have a script to turn these into paragraphized texts?

Some examples:
http://www.ietf.org/rfc/rfc793.txt (RFC: TCP)
http://www.gutenberg.org/files/345/345.txt (Dracula from Gutenberg) (Yes, I know there is an HTML version of that one.)

[Malakai]

Hey guys im tryin to grab a print version using the advanced version but i need to set the url.replace to change two things in the url.

here an example of the original url

http://www.dpreview.com/reviews/olympusep1/?from=rss

this is the url for the print version

http://www.dpreview.com/reviews/prin...iew=OlympusEP1

how do i get it to remove the /?from=rss at the end

This is what i currently have

def print_version(self, url):
return url.replace('http://www.dpreview.com/reviews/', 'http://www.dpreview.com/reviews/print.asp?review=')

[kiklop74]

Originally Posted by Malakai

Hey guys im tryin to grab a print version using the advanced version but i need to set the url.replace to change two things in the url.

here an example of the original url

http://www.dpreview.com/reviews/olympusep1/?from=rss

this is the url for the print version

http://www.dpreview.com/reviews/prin...iew=OlympusEP1

how do i get it to remove the /?from=rss at the end

This is what i currently have

def print_version(self, url):
return url.replace('http://www.dpreview.com/reviews/', 'http://www.dpreview.com/reviews/print.asp?review=')

Try this:

    def print_version(self, url):
        baseurl = url.rpartition('/?')[0]
        turl = baseurl.partition('/reviews/')[2]
        return 'http://www.dpreview.com/reviews/print.asp?review=' + turl

[Malakai]

Thank you so much for that, only thing missing are the pictures lol.
How do i retain those in the finished epub.

[jbambridge]

Problem parsing guardian rss feed:

I have tried to update the Guardian Recipe to fix some problems with changes in the web site etc. I am almost there, but I am hitting the odd article that causes the following errors in ebook-convert:

Parsing feed_0/article_7/index.html ...
Traceback (most recent call last):
File "cli.py", line 254, in <module>
File "cli.py", line 246, in main
File "calibre\ebooks\conversion\plumber.pyo", line 657, in run
File "calibre\ebooks\conversion\plumber.pyo", line 761, in create_oebbook
File "calibre\ebooks\oeb\reader.pyo", line 72, in __call__
File "calibre\ebooks\oeb\reader.pyo", line 588, in _all_from_opf
File "calibre\ebooks\oeb\reader.pyo", line 243, in _manifest_from_opf
File "calibre\ebooks\oeb\reader.pyo", line 176, in _manifest_add_missing
File "calibre\ebooks\oeb\base.pyo", line 988, in fget
File "calibre\ebooks\oeb\base.pyo", line 917, in _parse_xhtml
TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

The modified recipe is as follows:

#!/usr/bin/env python
__license__ = 'GPL v3'
__copyright__ = '2008, Kovid Goyal kovid@kovidgoyal.net'
__docformat__ = 'restructuredtext en'

'''
www.guardian.co.uk
'''

from calibre.web.feeds.news import BasicNewsRecipe

class Guardian(BasicNewsRecipe):

title = u'My Guardian'
language = _('English')
encoding = 'utf-8'
oldest_article = 7
max_articles_per_feed = 20
remove_javascript = True
simultaneous_downloads = 1
use_embedded_content = False
recursions = 0
filter_regexps = [r'\.g\.doubleclick\.net']

timefmt = ' [%a, %d %b %Y]'

keep_only_tags = [dict(id=['article-wrapper', 'main-article-info'])]




no_stylesheets = True
extra_css = 'h2 {font-size: medium;} \n h1 {text-align: left;}'


feeds = [
('Front Page', 'http://feeds.guardian.co.uk/theguardian/rss'),
# ('UK', 'http://feeds.guardian.co.uk/theguardian/uk/rss'),
# ('Business', 'http://www.guardian.co.uk/business/rss'),
# ('Politics', 'http://feeds.guardian.co.uk/theguardian/politics/rss'),
# ('Culture', 'http://feeds.guardian.co.uk/theguardian/culture/rss'),
# ('Money', 'http://feeds.guardian.co.uk/theguardian/money/rss'),
# ('Life & Style', 'http://feeds.guardian.co.uk/theguardian/lifeandstyle/rss'),
# ('Travel', 'http://feeds.guardian.co.uk/theguardian/travel/rss'),
# ('Environment', 'http://feeds.guardian.co.uk/theguardian/environment/rss')
]

def print_version(self, url):
return url + '/print'

Any ideas what the error means?

John

[jbambridge]

One extra thought:

Checking:

File "calibre\ebooks\oeb\base.pyo", line 917, in _parse_xhtml

in the source code shows that this is a part of the code that removes empty <a></a> tags. This is indeed the case on the example I gave where the publisher has left a strange link in the text.

Adding a

PHP Code:

remove_tags = [dict(name='a')] 


is a work around, although this also destroys valid <a> tags.

My PHP is not up to fixing the _parse_xhtml code myself though.

Can anyone suggest a better work around (that doesn't delete any valid content) or a fix to the PHP code?

John

P.S. I've attached the offending article as an example of the empty <a> tags. index.txt is after porcessing by the recipe and problem.txt is the original html file.

[kovidgoyal]

Will be fixed in next release.

[kiklop74]

I tried creating recipe for a new version and was not able to make conversion_options work.

Is it operational at all?

This is what I tried:

    conversion_options = {  'tags':'aa,bb'
                          , 'publisher': 'pub'
                          , 'comments':  'desc'
                          , 'language': 'en'
                          }

[kovidgoyal]

EDIT: Actually, looking at the code, it should be.

[kiklop74]

Originally Posted by kovidgoyal

EDIT: Actually, looking at the code, it should be.

well it does not work. Do you want issue for this?

[kovidgoyal]

Originally Posted by kiklop74

well it does not work. Do you want issue for this?

Never mind I'm looking at it now.

[jj2me]

Since I couldn't find the Smithsonian Magazine in a search of this thread, and it's my sister's favorite magazine, I humbly submit this bare minimum effort (don't know Python) in case anyone else might like it and doesn't mind skipping over some poor formatting.

It's merely the RSS assembling from this page. Note that I set oldest_article = 30 for this monthly magazine. Change as you see fit.

[AprilHare]

Attached is the errors I got when I tried to download the Sydney Morning Herald - too long for a simple post..