What are my options for reading daily newspapers on Iliad?

[jackbrown]

So I've had my Iliad since last summer. I'm reasonably happy with reading PDF'd books on it, either downloaded or created from my own library.

Irex clearly envisioned the device as a newspaper reader as well, but so far no English language newspapers seem to have signed up for IDS (if the WashPost/NYTimes/UKIndependent/WSJ offered an IDS subscription I'd sign up in a heartbeat). A few of our more technically minded developers here at Mobileread seem to have created scripts which download a news site (I'm thinking of the BBC script I tried in december) on a one-time basis.

But with all the RSS feeds, and free newspaper websites available, why hasn't anybody come up with a hack that simply downloads a paper or a news site automatically onto the Iliad when you connect it? Would this involve some significant reworking of the OS?

I submit that the device would be significantly more useful to a lot of us if somebody with some programming skills could make the device pull the BBC site down automatically, for instance (or even better, though perhaps more difficult, the NYT).

[Hadrien]

We're currently working on this for Feedbooks. Right now we're beta testing a simple RSS feed to PDF, mostly working on improving the support for the HTML to LaTeX parser.

We'll offer an easy way to create e-newspapers with multiple RSS feeds + elements such as weather forecast, sudoku etc...

We're also working on a small app to autosynchronize those newspapers on both the iLiad and the PRS-500.

If you're looking for a way to pull content from a website, you should try Dapper: http://www.dapper.net
You can easily extract content from any website using Dapper, create RSS feeds out of it and then use these feeds as elements for your newspaper.

[jackbrown]

I tried out the RSS->PDF link on FeedBooks: for those who haven't tried it yet, it works extremely well, formatting an Ebook version of an RSS feed perfectly for the Iliad. The main problem is of course that no newspapers (in English or Spanish at least, the ones I read) offer full articles in their RSS feeds. So you end up with a nice series of opening paragraphs.

Dapper looks interesting; I will report my results after playing with it a bit more.

The autosynchronize app that you are working on sounds WONDERFUL; I will beta test it any time you guys are ready to release something. Meanwhile I am continuing my occasional emailing campaign to my favorite newspapers to get them to pay attention to eink devices.

[Hadrien]

The best way newspapers could provide some support for e-ink devices would be with multiple feeds/subscriptions. Instead of getting the whole newspaper, you could get the financial section of one newspaper, and the world news section from an other newspaper. Simply doing the same stuff than a normal newspaper, but with e-ink, isn't the best solution here. An electronic newspaper should be much more flexible and support both custom content and content from both paper and online editions of these newspapers.

[geekraver]

Originally Posted by jackbrown

I tried out the RSS->PDF link on FeedBooks: for those who haven't tried it yet, it works extremely well, formatting an Ebook version of an RSS feed perfectly for the Iliad. The main problem is of course that no newspapers (in English or Spanish at least, the ones I read) offer full articles in their RSS feeds. So you end up with a nice series of opening paragraphs.

Dapper looks interesting; I will report my results after playing with it a bit more.

The autosynchronize app that you are working on sounds WONDERFUL; I will beta test it any time you guys are ready to release something. Meanwhile I am continuing my occasional emailing campaign to my favorite newspapers to get them to pay attention to eink devices.

web2book can get full articles. You can find out more about it in the sticky thread in the Sony Reader Content forum.

[Adam B.]

I have a lot of RSS feeds that I keep up on. Lately I've been using the Mobipocket desktop interface and fbreader on the iLiad. It downloads the full article as it appears in the rss feed, and looks great on the iLiad.

[Hadrien]

Originally Posted by Adam B.

I have a lot of RSS feeds that I keep up on. Lately I've been using the Mobipocket desktop interface and fbreader on the iLiad. It downloads the full article as it appears in the rss feed, and looks great on the iLiad.

Full article as it appears in the RSS feed ? The problem here is that most of the time, the full article isn't included in the RSS feed. That's why I use Dapper, it can easily extract any elements on a webpage. Since most of the news pages from a single website look the same, you can extract the information from there.

Web2book does a similar job, but not as advanced as Dapper and without a graphic user interface.

[jackbrown]

Tried out geekraver's web2book. Its a great idea but way too slow and too many operations involved to make it a daily habit.

Likewise Dapper--totally unusable for nontechnical users, and not very easy even for the somewhat technically minded.

what we need is push-button syncing of a newspaper on the iliad. and it ain't here yet.

[chrisv5]

Originally Posted by jackbrown

what we need is push-button syncing of a newspaper on the iliad. and it ain't here yet.

Wholeheartedly agree. This should be a commodity, not a challenge. While I personally enjoy the versatility of the device and love the partial openness, first and foremost it should be a tool.

[chrisv5]

Originally Posted by Adam B.

I have a lot of RSS feeds that I keep up on. Lately I've been using the Mobipocket desktop interface and fbreader on the iLiad. It downloads the full article as it appears in the rss feed, and looks great on the iLiad.

So, this should also work with the upcoming Mobipocket reader on the iLiad? If so, I am already very excited!

[jackbrown]

Good news on the newspaper front

I've come up with a solution that seems to work pretty well for more or less automatic updating of PDF newspapers on the Iliad. That's the biggest limitation so far: not a whole lot of my favorite papers produce a daily PDF edition.

Here's how I did it (this may be obvious for some people, but not for others):

I have a copy of Offline Explorer Enterprise,
and I downloaded a copy of the Iliad Companion Software ('ICS') from Irex.

First, I set up the ICS so that it automatically synchronizes with a folder on my desktop called Iliad. Inside that folder, it created a directory called:

\Iliad\My Iliad\Outbox\Internal\Newspapers

The Outbox is for anything you want to sync from computer->reader; Internal means the device's internal memory (you could have it sync to the CF card instead)

Next, I opened up Offline Explorer, which is a tool which lets you schedule automatic downloads of websites, or specific filetypes and filenames FROM a website.

Some PDF newspapers, like the Guardian, don't assign a new filename to each day's paper, but just keep the day's paper on a particular page, always with the same filename. That's a bit easier to handle. In that case, you do the following:
<File: New [Project]>
If there's a URL in the clipboard (you copied it from a webpage), that's automatically the URL and Project name for your autodownload. You Pick
<Level Limit-1> which means it doesn't follow all the URLs on a page past the first level.
Now, turn off all the <File Filters> except for <archives>, and in the <archives> submenu, uncheck everything but PDFs

Now, you go to the bottom, pick <scheduling>, and on the right, change <manually> to <daily, at a time shortly before you want to charge up your newspapers.

Close this dialog, and open <options> from the menu bar at the top. From <options> choose <advanced:File locations>
and browse to the same folder you picked in the ICS software:
\Iliad\My Iliad\Outbox\Internal\Newspapers\

-------

For newspapers which change the filename daily (ie each PDF filename includes the edition's date), it's a little more complicated.

Back in the original
Choose <URL Filters:Server>, and check Load files only within starting server,
Then in <filename> choose <Custom Filenames configuration>, <view included filenames> and in the Keyword box, type in as much of the filename as is consistent in every day's copy of the paper. For the Daily Telegraph, that's [telegraphpm_]

Now your computer should autodownload your papers to the ICS folder, and when you plug in the iliad, it autoupdates your papers.

I've got this working with the following papers:
Telegraph
Guardian
Toronto Star PM
El Universal (Mexico)
Al Khabar (Algeria)
Al Nahar in Lebanon
Al Quds in the UK

Unfortunately, most of the papers I like to read (NYT/Washpo/Independent/Masri Al Youm/Reforma) aren't available as PDFs, so I am still thinking about the Dapper/Web2Book solution for those....

[Hadrien]

Dapper updated their service and it's MUCH more user friendly. Oh and we're working on something on Feedbooks that could easily work directly on the iLiad itself: this way you could get your newspaper made out of our various "news widgets" (for the moment we have RSS and Sudoku) using the iLiad's wifi for example.

[Tommy]

Hi,

maybe this is the opportunity the perl-hack (getfeed.pl) I posted in the Scotty's "Daily Illadian" thread
What this hack does is to fetch one or more feeds and convert them into either an HTML or LaTeX file. Further, it is possible to retrieve the articles to which the feeds are pointing to. In addition to this, it is possible to specify "cutting-tags" (per feed of course), in order chop off annoying stuff (like ads, links referring to other articles, etc)

As this is maybe not exactly what the other posters have in mind, I also want to mention how this program is used: I have a (shell) script that first executes getfeed.pl, runs pdflatex again in order to get the toc right and then the resulting PDF is copied to my USB stick. At the train station I plug in this stick and execute a sript on the Iliad that copies myFeeds.pdf to the Iliad's news folder.
This routine proved quite handy, as there is no need to actually boot the Iliad in the mornings to get the file copied from my PC to the Iliad, but only once to copy the news from the USB stick and then I start reading.

Just as an appetiser, I attached today's edition of my "Daily Illadian"

Best regards,
Tommy

PS: yes, I'm running Linux But in principle it should be possible to use this on an MS-box, too, however some work would have to be invested (perl, latex), but still you wouldn't need to have any developing skills.

[jgrnt1]

I just got my Iliad yesterday. I've been experimenting with Mobipocket Reader Desktop. It pulls the full article on many RSS feeds and creates ebooks of them automatically. I looked at several newspaper sites, with mixed results -- some only pulled the first page of an article. Some added more junk text at the end of the article then the article itself contained. I also tried news organizations, like Reuters and the Associated Press, directly. AP did better than Reuters. Lastly, I tried USA Today, which pulled full articles with photos, which Mobipocket did a good job of converting to ebooks. USA Today, like most US newspapers, gets the majority of its news from AP and Reuters anyway. Being US based, I haven't tried iternational newspapers, but Mobipocket has done a pretty good job for me. I have attached an example.

[Hadrien]

Originally Posted by Tommy

Hi,

maybe this is the opportunity the perl-hack (getfeed.pl) I posted in the Scotty's "Daily Illadian" thread
What this hack does is to fetch one or more feeds and convert them into either an HTML or LaTeX file. Further, it is possible to retrieve the articles to which the feeds are pointing to. In addition to this, it is possible to specify "cutting-tags" (per feed of course), in order chop off annoying stuff (like ads, links referring to other articles, etc)

As this is maybe not exactly what the other posters have in mind, I also want to mention how this program is used: I have a (shell) script that first executes getfeed.pl, runs pdflatex again in order to get the toc right and then the resulting PDF is copied to my USB stick. At the train station I plug in this stick and execute a sript on the Iliad that copies myFeeds.pdf to the Iliad's news folder.
This routine proved quite handy, as there is no need to actually boot the Iliad in the mornings to get the file copied from my PC to the Iliad, but only once to copy the news from the USB stick and then I start reading.

Just as an appetiser, I attached today's edition of my "Daily Illadian"

Best regards,
Tommy

PS: yes, I'm running Linux But in principle it should be possible to use this on an MS-box, too, however some work would have to be invested (perl, latex), but still you wouldn't need to have any developing skills.

What we're working on with Feedbooks should work the same way with the possibility to add a few feeds to a single PDF files (+ some other stuff like sudokus, and we're working on additional stuff like weather forecast).
One pretty cool feature on the iLiad is its Wifi connection. We have a subscription feature on Feedbooks that generate an XML files with all the subscription informations for every user: this way, building an app that would work directly on the iLiad itself, and that would fetch the newspapers is going to be pretty easy.

The URL for this file is: http://www.feedbooks.com/user/newsstand/?user={yourlogin}&password={MD5 Hash of your password}
And the answer looks like something like that:

<newsstand>
<status>OK</status>
<subscription template="iLiad" type="Newspaper">
<title>Sudoku Maniac</title>
<url>http://www.feedbooks.com/news/newspaper2pdf/2?format=iliad</url>
</subscription>
<subscription template="Sony" type="Feed">
<title>Slashdot</title>
<url>http://www.feedbooks.com/news/rss2pdf/5?format=sony</url>
</subscription>
</newsstand>

First tag is for the status of your query: OK if you provided the right user/password.
And afterwards you get all the subscriptions from the website: the template, the type of content, the title of the content and the URL for this content.
This system is completely device agnostic, this way we can easily add support for new templates/contents/devices in the future.