Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Recipes

Notices

Reply
 
Thread Tools Search this Thread
Old 05-20-2011, 04:43 AM   #1
wintoid
Junior Member
wintoid began at the beginning.
 
Posts: 5
Karma: 10
Join Date: May 2011
Device: Kindle 3
vBulletin

Forums such as this one often use vBulletin as their software. vBulletin can make RSS feeds, so for example, you can read the Mobile Read forums at http://www.mobileread.com/forums/external.php?type=rss2 as an RSS feed.

Now, I can't find a recipe for a Calibre vBulletin forum anywhere, and when I've tried just the most basic steps, the results have been unsatisfactory.

Anyone got any ideas?
wintoid is offline   Reply With Quote
Old 05-20-2011, 10:40 AM   #2
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by wintoid View Post
Anyone got any ideas?
Add
Code:
    use_embedded_content= True
I tried it, and it works fine for me.
Starson17 is offline   Reply With Quote
Old 05-20-2011, 10:57 AM   #3
wintoid
Junior Member
wintoid began at the beginning.
 
Posts: 5
Karma: 10
Join Date: May 2011
Device: Kindle 3
Brilliant, thank you. That solves the first problem, and now I'm getting a decent looking RSS feed on my Kindle. The next problem is that I only get the first post to each thread. I have oldest_article=7 and max_articles_per_feed=100 which seem to be the defaults for a recipe, but I am not getting 7 days of feeds, or anything like 100 articles.

Sorry, I feel like such a leech!
wintoid is offline   Reply With Quote
Old 05-20-2011, 11:32 AM   #4
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by wintoid View Post
Brilliant, thank you. That solves the first problem, and now I'm getting a decent looking RSS feed on my Kindle. The next problem is that I only get the first post to each thread. I have oldest_article=7 and max_articles_per_feed=100 which seem to be the defaults for a recipe, but I am not getting 7 days of feeds, or anything like 100 articles.

Sorry, I feel like such a leech!
You're not a leech. It's OK to ask questions.

" use_embedded_content= True" Simply means to show you the feed page, not the linked articles. The feed page has only one post from each thread.

"oldest_article=" controls how old the article can be on the RSS feed page. Since all links are to recent posts, I suspect that all will be less than 7 days old.

"max_articles_per_feed=100" simply means if there are more than 100, don't show them. The feed appears to have only 15 articles.

If you want the linked pages (the threads) you need to write a recipe for them. You need to decide how far back you want to go and you'd need to write a multipage recipe, if you want multiple pages. Personally, I don't like forum recipes because if I'm interested enough to read it, I want to be able to post into the forum.

I assumed you just wanted to see the articles in the feed, then if you were interested, you'd click on the link within the recipe-ebook to open the forum in your browser, where you could read the entire thread.
Starson17 is offline   Reply With Quote
Old 05-20-2011, 11:51 AM   #5
wintoid
Junior Member
wintoid began at the beginning.
 
Posts: 5
Karma: 10
Join Date: May 2011
Device: Kindle 3
Thanks again. Yes, what I'm hoping for is to get all "posts" within the "thread". I've programmed C before but not Python, and am completely new to Calibre. Is there an example somewhere I can see that iterates through the "posts"?

Also, does "article" mean the actual main "threads" or the "posts" within the "threads"?

Thanks for all your help
wintoid is offline   Reply With Quote
Old 05-20-2011, 01:44 PM   #6
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by wintoid View Post
Thanks again. Yes, what I'm hoping for is to get all "posts" within the "thread". I've programmed C before but not Python, and am completely new to Calibre. Is there an example somewhere I can see that iterates through the "posts"?

Also, does "article" mean the actual main "threads" or the "posts" within the "threads"?

Thanks for all your help
Calibre terminology is that the "feed" is a link to a page that has links to "articles." I checked the links, and in this case the "article" is the first page in a new thread. Depending on whether there are any replies to the first post, there may be additional pages. You might find something useful in the builtin slashdot recipe. I know there are some other forum recipes, but I don't recall any names.

You don't need to iterate through posts. They will all be on the linked "article" page, although in a very active thread, if you run the recipe late, there might be additional pages you would need to follow to get everything. You will need to strip the stuff you don't want. Read the "Using News Recipes" sticky here, particularly at the end for some useful starting links.
Starson17 is offline   Reply With Quote
Old 05-21-2011, 03:37 AM   #7
wintoid
Junior Member
wintoid began at the beginning.
 
Posts: 5
Karma: 10
Join Date: May 2011
Device: Kindle 3
OK, just so we're talking about the same thing, the feed I really want to read on my Kindle is http://www.rangefinderforum.com/foru....php?type=rss2 which is a vBulletin-based forum, like mobileread itself.

My recipe currently looks like this:

class AdvancedUserRecipe1305962894(BasicNewsRecipe):
title = u'Rangefinder Forum'
oldest_article = 7
max_articles_per_feed = 100
use_embedded_content = True
feeds = [(u'RFF', u'http://www.rangefinderforum.com/forums/external.php?type=rss2')]

This works beautifully except that it only ever gets the first post in a forum thread. Are you saying that every response post will be treated as a new "page"? To date, Calibre has never fetched ANY response post.

Again, my thanks for your help.
wintoid is offline   Reply With Quote
Old 05-23-2011, 09:30 AM   #8
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by wintoid View Post
OK, just so we're talking about the same thing, the feed I really want to read on my Kindle is http://www.rangefinderforum.com/foru....php?type=rss2 which is a vBulletin-based forum, like mobileread itself.

My recipe currently looks like this:

class AdvancedUserRecipe1305962894(BasicNewsRecipe):
title = u'Rangefinder Forum'
oldest_article = 7
max_articles_per_feed = 100
use_embedded_content = True
feeds = [(u'RFF', u'http://www.rangefinderforum.com/forums/external.php?type=rss2')]

This works beautifully except that it only ever gets the first post in a forum thread. Are you saying that every response post will be treated as a new "page"? To date, Calibre has never fetched ANY response post.

Again, my thanks for your help.
I'm saying that this:
use_embedded_content = True
only shows what's on the feed page., and that page never has response posts. You need to change that line to False (which is where you started) and then write the recipe to deal with images and all the junk it pulls down. If you need it, you'll also need to write it to grab the second and third pages, etc.
Starson17 is offline   Reply With Quote
Old 05-23-2011, 11:44 AM   #9
wintoid
Junior Member
wintoid began at the beginning.
 
Posts: 5
Karma: 10
Join Date: May 2011
Device: Kindle 3
Great! Now my recipe looks like this:

class AdvancedUserRecipe1305902518(BasicNewsRecipe):
title = u'Rangefinder Forum'
oldest_article = 7
max_articles_per_feed = 100

feeds = [(u'RFF', u'http://www.rangefinderforum.com/forums/external.php?type=rss2')]
use_embedded_content= False
keep_only_tags = [dict(name='div', attrs={'id':'posts'})]

However, the problem now is that the RSS is only returning 15 articles, but how has it determined that 15 is "since I last read it"?
wintoid is offline   Reply With Quote
Old 05-23-2011, 02:25 PM   #10
Starson17
Wizard
Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.Starson17 can program the VCR without an owner's manual.
 
Posts: 4,004
Karma: 177841
Join Date: Dec 2009
Device: WinMo: IPAQ; Android: HTC HD2, Archos 7o; Java:Gravity T
Quote:
Originally Posted by wintoid View Post
Great! Now my recipe looks like this:

class AdvancedUserRecipe1305902518(BasicNewsRecipe):
title = u'Rangefinder Forum'
oldest_article = 7
max_articles_per_feed = 100

feeds = [(u'RFF', u'http://www.rangefinderforum.com/forums/external.php?type=rss2')]
use_embedded_content= False
keep_only_tags = [dict(name='div', attrs={'id':'posts'})]

However, the problem now is that the RSS is only returning 15 articles, but how has it determined that 15 is "since I last read it"?
An RSS feed page has links to web pages called "articles." In your case the article is the forum page with one or more posts. Your recipe gets everything on that article page. It gets all the articles on the feed. So, if there were 15 links on your feed page, you'd get 15 articles.

Trying to write a recipe to only retrieve unread articles will be much more complex. I don't even know if the RSS page has that info. I suspect it doesn't. Have you looked into the RSS aggregator recipes like Google Reader?
Starson17 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump


All times are GMT -4. The time now is 07:30 AM.


MobileRead.com is a privately owned, operated and funded community.