My first recipe YeY! I have a question though...

[audreypots]

Hi Guys,

I finally made my first recipe and so far so good (thanks calibre!). However, I was wondering if you guys can point me in the right direction (I really feel that my question is very stupid, but i read in this forum that the stupid question is the one that's not asked). First let me say that I have no idea what python is and a very basic knowledge of HTML, but I find this very interesting and the potential to be great! So I am trying to learn...

Anyways, my question is this, can you guys point me in the right direction on how I can include the images on the articles?

This is the link of the one I am working on:

http://blog.mysanantonio.com/spursnation/feed/

So far everything is okay, but the images are not showing and I have no idea why... I tried playing around with remove_tags_before/after and keep_only_tags/remove_only_tags but no success yet...

Sorry for the rant and I really do appreciate the help!

[Starson17]

Originally Posted by audreypots

Anyways, my question is this, can you guys point me in the right direction on how I can include the images on the articles?

Have you tried the Basic recipe? Does it keep images? All the keep or remove tags options you referred to act to eliminate tags. That's all they do, so you should start without any of those options and see if you get images. If not, the problem is elsewhere (It could be cookies, scripting problems, headers, login authentication, etc.). If you get images with the Basic recipe (which does not use any of the tag removal options), then you can look at how to get rid of other junk, without also inadvertently deleting images.

IOW, your first job is to figure out whether the images are missing because of something you are doing, or something the website is doing.

[louhike]

Give a try to keep_only_tags = [dict(name='div',attrs={'class':'post-contents clearfix'})])].
You should have everything you want with this parameter.

[audreypots]

Originally Posted by Starson17

Have you tried the Basic recipe? Does it keep images? All the keep or remove tags options you referred to act to eliminate tags. That's all they do, so you should start without any of those options and see if you get images. If not, the problem is elsewhere (It could be cookies, scripting problems, headers, login authentication, etc.). If you get images with the Basic recipe (which does not use any of the tag removal options), then you can look at how to get rid of other junk, without also inadvertently deleting images.

IOW, your first job is to figure out whether the images are missing because of something you are doing, or something the website is doing.

Hi and thank you very much for the help, I should have been more clear earlier, I actually started with just the basic recipe, only giving a title and the url of the feed (just like in the tutorial) and everything is fine I can read the whole article and as far as I can tell no extra junk from the site. Then I tried the Economist and I noticed that it include images so I tried playing around with it to try and add the image from the articles... that's why I used the tag removal options.

[audreypots]

Originally Posted by louhike

Give a try to keep_only_tags = [dict(name='div',attrs={'class':'post-contents clearfix'})])].
You should have everything you want with this parameter.

Hi thanks for the help! I think I tried this one already to make sure I tried it again and after fetching the whole article is gone, the article would be something like:

| Next | Section Menu | Main Menu |
This article was downloaded by calibre from http://blog.mysanantonio.com/spursna...-nbas-top-500/


| Section Menu | Main Menu |
| Next | Section Menu | Main Menu | Previous |
This article was downloaded by calibre from http://blog.mysanantonio.com/spursna...other-players/


| Section Menu | Main Menu |
| Next | Section Menu | Main Menu | Previous |

[Starson17]

Originally Posted by audreypots

Hi thanks for the help! I think I tried this one already to make sure I tried it again and after fetching the whole article is gone

Do you get images without any keep_only_tags or remove_tags, etc. in your recipe? If not, changing those functions will never produce images. You should always post your recipe if you want others to look at it.

[Starson17]

Originally Posted by audreypots

I actually started with just the basic recipe, only giving a title and the url of the feed (just like in the tutorial) and everything is fine

When you wrote "everything is fine" did you mean you had images?

If you didn't have images, then everything wasn't fine, but if you had images, then why would you have " tried playing around with it to try and add the image from the articles"?

If you did not have images with the basic recipe, the tag removal options won't improve anything.

[audreypots]

Originally Posted by Starson17

When you wrote "everything is fine" did you mean you had images?

If you didn't have images, then everything wasn't fine, but if you had images, then why would you have " tried playing around with it to try and add the image from the articles"?

If you did not have images with the basic recipe, the tag removal options won't improve anything.

Actually no, there was no image with the basic recipe, but I was able to read the whole article so I thought it was fine So if the tag removal options won't improve anything, what can I use/do to add the images? Again thanks for the help and enjoy the rest of your day!

[Starson17]

Originally Posted by audreypots

Actually no, there was no image with the basic recipe, but I was able to read the whole article so I thought it was fine So if the tag removal options won't improve anything, what can I use/do to add the images? Again thanks for the help and enjoy the rest of your day!

Post your recipe. Use CODE and SPOILER tags. I'll take a look. As to what you can do to fix it, you have to find out the problem. Now we know it's the site, and not your recipe removing a tag that contains the image. As to what the problem is: "(It could be cookies, scripting problems, headers, login authentication, etc.)."

[audreypots]

Originally Posted by Starson17

Post your recipe. Use CODE and SPOILER tags. I'll take a look. As to what you can do to fix it, you have to find out the problem. Now we know it's the site, and not your recipe removing a tag that contains the image. As to what the problem is: "(It could be cookies, scripting problems, headers, login authentication, etc.)."

Hi,

This is my code, I am afraid it's very basic:

Spoiler:

class AdvancedUserRecipe1301845915(BasicNewsRecipe):
    title          = u'My San Antonio Spurs'
    oldest_article = 1
    max_articles_per_feed = 100
    cover_url           = 'http://blog.mysanantonio.com/spursnation/wp-content/themes/niche-site-spurs/images/logo.png'

    feeds          = [(u'My San Antonio Spurs', u'http://blog.mysanantonio.com/spursnation/feed/')]

    #keep_only_tags = [dict(name='div',attrs={'class':'post-contents clearfix'})]

[Starson17]

Originally Posted by audreypots

Hi,

This is my code, I am afraid it's very basic:

That's OK, 1) I prefer to see what you're actually using, 2) it saves me time in organizing the basic structure, and 3) I'm sure you really are interested enough to post.

I ran your recipe and tracked it back at least far enough to see that the site is sensitive to all sorts of issues. If you turn off cookies in Firefox, or block the cookie in TamperData, you get no images. If you send no UserAgent, you get no images, etc. i suspect it may also be sensitive to other headers, like the accept header, etc.

Normally, the recipe system will provide basic cookie handling and it sends a default UserAgent. Something else is likely to be the problem. I had a site that needed an Accept header that Calibre was not sending to get past the Bad Behavior module.

I regret that I don't have time to solve the problem for you. Search for some of my posts on Accept headers, Bad Behavior, cookies, etc. to see how to track the HTTP handshaking, cookies and headers. You would need to see what Calibre sends, match that to the minimum that the site finds acceptable.

[audreypots]

Originally Posted by Starson17

That's OK, 1) I prefer to see what you're actually using, 2) it saves me time in organizing the basic structure, and 3) I'm sure you really are interested enough to post.

I ran your recipe and tracked it back at least far enough to see that the site is sensitive to all sorts of issues. If you turn off cookies in Firefox, or block the cookie in TamperData, you get no images. If you send no UserAgent, you get no images, etc. i suspect it may also be sensitive to other headers, like the accept header, etc.

Normally, the recipe system will provide basic cookie handling and it sends a default UserAgent. Something else is likely to be the problem. I had a site that needed an Accept header that Calibre was not sending to get past the Bad Behavior module.

I regret that I don't have time to solve the problem for you. Search for some of my posts on Accept headers, Bad Behavior, cookies, etc. to see how to track the HTTP handshaking, cookies and headers. You would need to see what Calibre sends, match that to the minimum that the site finds acceptable.

Thank you so much! Please no regrets (i do not know the exact phrase to reply, but I hope you get my point), as I wasted some of your time already. All I really need is a little nudge on the right the direction, a few days ago I don't even know the problem now I can concentrate!

Again many many many thanks!