Custom recipes (archive, read-only)

[Starson17]

In case anyone is interested:
I've begun reading Tom's Hardware. The recipe says a username and password is needed, but AFAICT, it's not. The RSS feeds directly from the site don't seem to ask for a login, and entering a random username and password into the calibre recipe seems to work fine.

[Abelturd]

Custom recipe for iLiteratura.cz, a czech website focusing on literature, book reviews, culture.

[Starson17]

The wife asks me to request a recipe (code) for recipes (food) from Epicurious:
http://www.epicurious.com/services/rss

She particularly wants the Healthy Recipes, New Recipes, Latest Features and Blogs (the first four feeds).
Thanks!

Edit: I am slowly making progress at doing this myself. I wouldn't have posted the request, except that it was beyond my skills. Of course, with sufficient effort, skills tend to improve.

I hope no one has spent too much time on it, although I can't yet determine if I will succeed.

Question: one page has this code:

            <div class='padTop14' id="recipeIntroText">
                <span id="truncatedText">Cooking a large piece of salmon fillet couldn’t be easier and helps ensure that the fish stays flaky and moist. The crushed caraway seeds in the...</span>
                <span class="greenLnk"><a class="greenLnk" onclick="swapIntroText(event);">more</a> ›</span>
                <script>
                        var recipeIntroText = "Cooking a large piece of salmon fillet couldn\u0092t be easier and helps ensure that the fish stays flaky and moist. The crushed caraway seeds in the glaze add a subtle twist to the classic honey and mustard combination, which really complements the full flavor of the salmon.";
                </script>
            </div>

Can anyone suggest how to replace the truncatedText with the contents of the var recipeIntroText?

Thanks.

[kiklop74]

Originally Posted by Starson17

In case anyone is interested:
I've begun reading Tom's Hardware. The recipe says a username and password is needed, but AFAICT, it's not. The RSS feeds directly from the site don't seem to ask for a login, and entering a random username and password into the calibre recipe seems to work fine.

Login is required for multipage articles. With login you get access to the print version. At least it was like that last year.

[Starson17]

Originally Posted by kiklop74

Login is required for multipage articles.

That explains it. Thanks.

[lorenzov]

here is a start for your epicurious recipe. if you look at the adventuregamers recipe there is an example of how to resolve multiple pages, however if there is a print version it is more likely to make things easier!

i reccommend to have a look at the docs on the wiki to see some other useful examples:
http://bugs.calibre-ebook.com/wiki/RecipeTips

[Starson17]

Originally Posted by lorenzov

here is a start for your epicurious recipe. if you look at the adventuregamers recipe there is an example of how to resolve multiple pages, however if there is a print version it is more likely to make things easier!

i reccommend to have a look at the docs on the wiki to see some other useful examples:
http://bugs.calibre-ebook.com/wiki/RecipeTips

Thank you. I had a semi-working recipe, but yours fixed the problem I referenced above and gave me more useful tips. The keep_only_tags and remove_tags I'd already figured out, plus parts of your recipe really improved the output. I'll head over to the wiki - I had read the User Manual, but had forgotten the wiki. It's really helpful.

Edit: If I understand your recipe, you figured out how to build the link to printer friendly version of the food recipes. That solves the problem above by avoiding the problem in the html code of the non-printer friendly version. However, in the blog links, I couldn't find a printer friendly version, so I need to either fix the problem or handle the two feeds differently. .... I'm off to read more sample recipes.

[Starson17]

Originally Posted by kiklop74

Login is required for multipage articles. With login you get access to the print version. At least it was like that last year.

I did go back and look at the output. I couldn't find any missing pages from multipage articles or anything else missing in the output created using a fake name and password. I don't know if they've changed their policy, or if I missed something I would have seen by creating a user/pass.

[kiklop74]

The multi page thing applies to the Articles, not news. On Tom's Hardware articles tend to have more than one page and on such pages you will see that the print link is disabled and leads you to the login form.

For example see:

http://www.tomshardware.com/reviews/...-ssd,2550.html

However they do not protect the print URL itself so the recipe will work even without correct login credentials.

I will not change the recipe to honor the spirit of intended usage.

[kiklop74]

Originally Posted by exdream

Please help!
I'm trying to figure out a recipe for http://szmobil.sueddeutsche.de/.

Try this in your recipe:

import urllib

....


    def get_browser(self):
        br = BasicNewsRecipe.get_browser()
        br.open('http://szmobil.sueddeutsche.de/login.php')
        if self.username is not None and self.password is not None:
            data = urllib.urlencode({ 'username':self.username
                                     ,'password':self.password
                                     ,'submit':'Login'
                                     ,'id':'0'
                                   })
            br.open('http://szmobil.sueddeutsche.de/login.php',data)
        return br

[exdream]

Originally Posted by kiklop74

Try this in your recipe:
[CODE] import urllib ....

kiklop74 - many, many thanks for your efforts, but (I nearly don't have the heart to say it) it still doesn't work. Same result: downloaded articles are login-page I hope you are not running out of ideas by now. I'm going to klick your donate-link if you find a working solution.

Greetings,
Gero

[kiklop74]

Originally Posted by exdream

kiklop74 - many, many thanks for your efforts, but (I nearly don't have the heart to say it) it still doesn't work. Same result: downloaded articles are login-page I hope you are not running out of ideas by now. I'm going to klick your donate-link if you find a working solution.

Greetings,
Gero

Could you send me your credentials for this site in private mail so that I can test this at home?

[Starson17]

Originally Posted by kiklop74

The multi page thing applies to the Articles, not news. On Tom's Hardware articles tend to have more than one page and on such pages you will see that the print link is disabled and leads you to the login form.

Thanks for the explanation.

[jackietreehorn]

I am trying to create a custom recipe for the Turkish newspaper site Radikal. Here is my code:

class AdvancedUserRecipe1265645305(BasicNewsRecipe):
    title          = u'Radikal'
    language = 'tr'
    oldest_article = 5
    max_articles_per_feed = 5
    cover_url = 'http://www.radikal.com.tr/D/i/1/V2/radikal_logo.jpg'
    feeds          = [(u'Yazarlar', u'http://www.radikal.com.tr/d/rss/RssYazarlar.xml')]

    def print_version(self, url):
      return url.replace('aType=RadikalYazarYazisi', 'aType=HaberYazdir')

I use the following command: "ebook-convert radikal.recipe output -vv". The folder "output" is created, however I get bunch of errors and my created index.html is almost empty. I have the following issues:

1- Could not fetch link http://www.radikal.com.tr/Default.aspx?aType=HaberYazdir&Ar
ticleID=979148
Traceback (most recent call last):
File "site-packages\calibre\web\fetch\simple.py", line 442, in process_links
File "site-packages\calibre\web\feeds\news.py", line 599, in _postprocess_html
AttributeError: 'NoneType' object has no attribute 'insert'
http://www.radikal.com.tr/Default.as...ticleID=979148 saved
to c:\docume~1\engin\locals~1\temp\calibre_0.6.37_nba ozi_plumber\feed_0\article_
3\Default.xhtml

There is a created issue for this "AttributeError" (http://bugs.calibre-ebook.com/ticket/1270). However, my page does not have the DOCTYPE issue. I do however get validation errors if I try to validate the page.


2- print_version function does not work for all the URLs. For instance:
- 7% Article download failed: u'\u0130\u015fsiz aileye ayda 300 lira'
Failed to download article: ... http://www.radikal.com.tr/Default.aspx?aType=RadikalYazarYazisi&ArticleID=979171

I would expect "RadikalYazarYazisi" to be replaced with "HaberYazdir" here.

3- On some pages I get " Exception: Could not fetch article. Run with -vv to see the reason", even though I have the "-vv" option. How do I set the debug mode on?

[DoctorOhh]

Originally Posted by jackietreehorn

I am trying to create a custom recipe for the Turkish newspaper site Radikal. Here is my code:

class AdvancedUserRecipe1265645305(BasicNewsRecipe):
    title          = u'Radikal'
    language = 'tr'
    oldest_article = 5
    max_articles_per_feed = 5
    cover_url = 'http://www.radikal.com.tr/D/i/1/V2/radikal_logo.jpg'
    feeds          = [(u'Yazarlar', u'http://www.radikal.com.tr/d/rss/RssYazarlar.xml')]

    def print_version(self, url):
      return url.replace('aType=RadikalYazarYazisi', 'aType=HaberYazdir')

I use the following command: "ebook-convert radikal.recipe output -vv". The folder "output" is created, however I get bunch of errors and my created index.html is almost empty. I have the following issues:



2- print_version function does not work for all the URLs. For instance:
- 7% Article download failed: u'\u0130\u015fsiz aileye ayda 300 lira'
Failed to download article: ... http://www.radikal.com.tr/Default.aspx?aType=RadikalYazarYazisi&ArticleID=979171

I would expect "RadikalYazarYazisi" to be replaced with "HaberYazdir" here.

I can't help with 1 or 3 and I am not versed in all of the ways of getting the print page but I am relatively sure the below code will work.

    def print_version(self, url):
        return url.replace('http://www.radikal.com.tr/Default.aspx?aType=HaberYazdir', 'http://www.radikal.com.tr/Default.aspx?aType=RadikalYazarYazisi')