Kindle 2 and News Feeds

[Bookworm88]

I'm trying to read the Observer on my Kindle 2 using Calibre to download the RSS feeds. When I use Calibre's viewer I can see all the articles from the paper in the approx. 600mb book it creates. When I upload it to my Kindle either by emailing it or directly through the USB cable I only get less than twenty articles in only two sections of the paper. If I look at the Jobs List after the upload it shows 4 jobs:
1)Get device info.
2)Get list of books on device
3)Upload 1 books to device: Charlotte Observer
4)Send metadata to device

All of these jobs take 0m 0s each and say they're 100% complete. I'm attaching a text file of the news fetching job in case it's helpful.
Any help with this would be appreciated.

[Bookworm88]

Hmmmm, 74 views and no one has any idea what the problem is here?

[GRiker]

Try posting your recipe, that might be a better starting place.

G

[Bookworm88]

Thanks for the reply. I didn't post the recipe because it was downloading correctly to the Calibre reader but not transferring everything to my Kindle. At any rate if I'm looking in the right place here is the recipe I created for the Observer.

class AdvancedUserRecipe1251384835(BasicNewsRecipe):
title = u'Charlotte Observer'
oldest_article = 7
max_articles_per_feed = 100

feeds = [(u'Local News', u'http://www.charlotteobserver.com/531/index.rss'), (u'Breaking Local News', u'http://www.charlotteobserver.com/breaking/index.rss'), (u'Politics', u'http://www.charlotteobserver.com/143/index.rss'), (u'National', u'http://www.charlotteobserver.com/116/index.rss'), (u'World', u'http://www.charlotteobserver.com/235/index.rss'), (u'Business', u'http://www.charlotteobserver.com/123/index.rss'), (u'Banking', u'http://www.charlotteobserver.com/138/index.rss'), (u'Sports', u'http://www.charlotteobserver.com/532/index.rss'), (u'Carolina Panthers', u'http://www.charlotteobserver.com/290/index.rss'), (u'High School Sports', u'http://www.charlotteobserver.com/preps/index.rss'), (u'Entertainment', u'http://www.charlotteobserver.com/122/index.rss'), (u'TV', u'http://www.charlotteobserver.com/167/index.rss')]

[GRiker]

Bookworm88,

I ran your recipe, changing max_articles_per_feed to 5, so that I could get a general sense of what was happening without downloading the entire history of Charlotte. I fetched articles in 6 categories - Local News, Breaking Local News, Politics, National, World and Business. All of the categories except Business downloaded 5 articles, except Business which had 1. This highlights an important notion to be aware of:

If an included RSS feed is empty at the time you run the recipe, you won't get any content in that feed. This can happen for a number of reasons, including a timeout from the remote server, remote server maintenance, or an error of any sort. Often, running the recipe again a few minutes later works.

For the feeds that generated no content (e.g., Sports, Carolina Panthers) a quick look at the RSS feeds shows different URLS than the ones you're using. You should check each of the feeds from this page to make sure yours are the same.

I would encourage you to reduce max_articles_per_feed to a lower number, and try again. Other than needing to confirm your feed URLs, I don't see anything wrong with your recipe.

G

[Bookworm88]

I appreciate your effort to troubleshoot this problem for me. The problem I'm experiencing is not really with the recipe, I don't think. See, when I run it the resulting ebook that I get reads fine in the Calibre viewer. The problem is when I transfer it to the Kindle. For some reason very little of it is transferring. I'll try your suggestions and see if it helps. Thanks again.

[GRiker]

Your original post said that your output file was 600MB. If that's correct, it's a gigantic file, which may be breaking some internal limits of the Kindle or calibre. By reducing the number of articles fetched from each feed, if you can get the file down to a more reasonable size, my sense is that it will work.

By way of comparison, a typical Sunday NYTimes download purchased from Amazon, with 200+ articles, is ~3MB.

G

[jbambridge]

I get a similar problem with the new standard recipe for The Guardian on my Kindle DX. All of the articles are being downloaded but the DX Table of Contents only shows a few in the first section (sometimes it shows two sections, but never the complete TOC.

I know the recipe is working as if I keep paging past the last article in the TOC, all of the rest of the articles are there, just not listed in the TOC.

Other recipes (eg The Telegraph and The Economist) have compete TOCs and work perfectly on the DX.

Any ideas?

[Bookworm88]

Originally Posted by GRiker

Your original post said that your output file was 600MB.

I don't know what I was smoking when I posted that. It is 649k not 600 mb.

[Bookworm88]

Originally Posted by jbambridge

I get a similar problem with the new standard recipe for The Guardian on my Kindle DX. All of the articles are being downloaded but the DX Table of Contents only shows a few in the first section (sometimes it shows two sections, but never the complete TOC.

I know the recipe is working as if I keep paging past the last article in the TOC, all of the rest of the articles are there, just not listed in the TOC.

Other recipes (eg The Telegraph and The Economist) have compete TOCs and work perfectly on the DX.

Any ideas?

Sounds like exactly what I experienced The Calibre viewer TOC shows everything and all articles are accessible but the K2 TOC only shows 15-20 articles but I can only view what is showing in the K2 TOC.

[GRiker]

jbambridge + Bookworm88:
Can we clarify whether you're talking about Kindle's built-in TOC mechanism, or the TOC that calibre appends to the end of the downloaded content?

For periodicals on the Kindle, the TOC you should be using is the native Kindle TOC - the one accessed by pressing the 'View Sections List' or 'View Articles List' with the 5-way joystick, at the bottom of the screen. Doing so takes you to the list of articles in each section, from which you can easily navigate to the section/article of your choice.

Calibre also creates TOCs for non-Kindle readers, appended to the end of the file, but this is redundant for a Kindle. There is a switch in the Mobi output to suppress the TOC generation, which I'd suggest that Kindle users enable.

G

[jbambridge]

I am talking about the Kindle's built in TOC mechanism. Using View Articles List only shows one (or at most two) sections with 10-12 entries. The rest are not shown in the Kindle TOC.

John

[GRiker]

jbambridge, thanks for confirming. Could you post your recipe so I could try it?

G

[jbambridge]

jbambridge, thanks for confirming. Could you post your recipe so I could try it?

I am using the standard recipe for "The Guardian" that comes with 6.10.

[GRiker]

I've confirmed what you've reported, and I will report back when I learn what happened to the missing TOC entries.

G