Full Articles via RSS

[jotheman]

Hello,

Mobipocket seems to be the only software that can use any rss feed to fetch not only the headlines included there but the full corresponding articles.

I would like to know if there is other software out there which can do this, especially for Mac OS X? Or any other way to get it done?

I want to read my news on an eReader that doesn't support Mobipocket format and without having to use Windows for conversion.

Please share your knowledge! Thanks!


jo.

[kovidgoyal]

calibre

[jotheman]

Thanks for the quick reply, kovidgoyal!

I downloaded Calibre and did some tests. (It was kind of hard to find via a google search, even though I knew what I was searching for.)

It definitely is a nice peace of software and I am glad that there are people creating tools like this, even making them available for free on the web!


If I may, I would like to do a quick summary of my findings:

- Nice layout and design, even though the buttons are a bit large and window size was kind of a problem. I maximized it and was then unable to make it small again, because the "resize triangle" was out of reach, somewhere outside the desktop area.

- RSS fetching workes fine, but can take up to 2 minutes. That's not a huge problem, Mobipocket was much faster though and it even included images.

- Calibre only supports LRS format, sadly making it quite useless for people owning anything other than a Sony Reader.


I like it, but since I don't plan on buying a Sony device (just placed an order for a BeBook / Hanlin V3) or want to have to do a conversion of every file that Calibre creates, I'm afraid I need to keep looking for alternatives. At least until Calibre will support other formats or allows exporting / printing files to pdf for example.


Are there more alternatives??


.

[kovidgoyal]

Use feeds2disk (part of calibre) from the commandline, it will download and convert your feeds into a set of linked HTML files. You can then convert them to your favorite format using its conversion tools.

[jotheman]

Dear Kovid,

thanks for you always blazing fast replies.

HTML files would be great, I think I could just use them directly (if linked html files work on the Hanlin - have to wait for the device arriving to check that).

I'm really excited to hear that what I need can be done using Calibre, but I don't know how to use python scripts and even if I got it to work, how could I automate it? Could I create a script that I just had to click on once daily to get my news, or how does it work?

Are you planning on integrating this into the GUI? Have Calibre natively support multiple formats and download full articles from RSS - man, that would be a real killer-app!

I see light on the horizon. Thanks again,



jo.

[pilotbob]

Originally Posted by jotheman

Mobipocket seems to be the only software that can use any rss feed to fetch not only the headlines included there but the full corresponding articles.

Alot of that depends on the feed. Most RSS feeds have full text... but many only include headlines and a snipit of the full text. This is because they want you to visit the site so you can see all the wonderful adds and you can be counted.

BOb

[kovidgoyal]

Eventually calibre will get support for more output formats, when it does, adding RSS - to whatever format is trivial.

[jotheman]

Originally Posted by pilotbob

Alot of that depends on the feed. Most RSS feeds have full text... but many only include headlines and a snipit of the full text. This is because they want you to visit the site so you can see all the wonderful adds and you can be counted.

@ BOb
I am aware of that tactic and I's annoying me. That's why I was really excited when I noticed that Mobipocket fetches full articles, even if the feed is abbreviated. I mostly read news from state-run news stations. There isn't even advertising on those pages and I pay for their financing anyway - everyone owning a TV or Radio here in Germany has to pay the "GEZ"-fee for public radio/tv. It's about 17 € a month. So I don't see why I should read their articles online and not take them with me on my eBook reader.

So I know it's possible and I'd like to see it on Mac.

Don't know why this isn't a bigger issue with people. The difference is dramatic - news snipets vs. full articles!


@ Kovid
When are you planning to extend the list of supported formats? If the script is already there it can't be that difficult to implement into the GUI, right?

I decided to support your work. I think I saw a donation button somewhere on that page...

Cheers,


jo.

[jotheman]

BTW, what is the best way of giving feedback for calibre?

[kovidgoyal]

well i'm going to add epub support next (hopefully both the sony reader and the cybook will support it soon). Unfortunately, since I only work part itme on calibre, I can't give you a firm timeline, it all depends on my regular work load.

As for feedback, you should open tickets at http://calibre.kovidgoyal.net/newticket

[jotheman]

Thanks a lot, Kovid.
I appreciate your work. Already created some tickets, hope you don't mind.


If anyone else has an idea how to fetch full articles from abbreviated RSS feeds on Mac OS X, please contribute!

Cheers,


jo.

[pilotbob]

Originally Posted by jotheman

Thanks a lot, Kovid.
I appreciate your work. Already created some tickets, hope you don't mind.


If anyone else has an idea how to fetch full articles from abbreviated RSS feeds on Mac OS X, please contribute!

I'm wondering if perhaps Mobi is not using the RSS feeds from some of these sites and just pulling down the HTML. I am pretty sure that is what web2lrf is doing. Using RSS feeds would give a much cleaner result... of course, the RSS feed needs to include the full text of articles for this to work.

I personally don't subscribe to any RSS feeds that don't send the full text down.

BOb

[kovidgoyal]

feeds2lrf does both

[jotheman]

I guess mobipocket's way of doing things is probably quide similar to calibre's. But calibre rules for two reasons:

- it's free and multiplatform
- it lets you tweak the way, feeds are fetched and displayed

I created 'recipies' for several of the feeds I need and after some tweaking here and there they now work like a dream and look really nice. Even pictures are included, which I didn't notice earlier. Kudos to Kovid.

Now I just have to wait for him to (hopefully soon) add support for other formats and/or add a printing feature (easily create PDFs).

What the guys developing Books.app for Mac OS X are doing sounds interesting, too. The next version of their app will include eBook management (right now it's just for managing your on-paper library). Maybe they will have a "news" feature with RSS support as well. Maybe even one as powerful as calibre's. And let's wait what happens at the next Macworld...

Cheers



jo.

[jotheman]

Kovid,

is it possible to tweak feed fetching for sites that not only change the parts of the article address before the article file in printing mode (like BBC), but also add stuff at the end?

I'm trying to tweak the feeds from the German webzine Telepolis. One of the feeds offered is http://www.heise.de/tp/news.rdf

An example would be (again):
http://www.heise.de/tp/r4/artikel/28/28035/1.html,
which changes to
http://www.heise.de/bin/tp/issue/r4/...035&mode=print
for printing mode.

Help appreciated,

Cheers


jo.