Feedbooks.com iLiad Downloader (iNewsStand)

[hansel]

Originally Posted by nrapallo

I'm interested!
[...]But, while you are at it would a Mobileread.com download app be possible using the same framework as Feedbooks.com?

Hey, that would be cool! Are all those uploaded books archived somewhere in a way a computer can understand?

PS: a.f.a.i.k. the Iliad already can surf the internet (sort of) with minimo.

[nrapallo]

Originally Posted by hansel

Hey, that would be cool! Are all those uploaded books archived somewhere in a way a computer can understand?

PS: a.f.a.i.k. the Iliad already can surf the internet (sort of) with minimo.

The Mobipocket Download Guide just lists the all the .prc/.mobi attachments available for download at Mobileread.com filtering out all the other formats or uploads.

If you explode the .mobi that gets downloaded when you access the Mobipocket Download Guide under the E-Books menu item at the top of the page in the Blue Bar, then you will see exactly how this is accomplished.

In particular, when referenced, a URL is re-directed to the exact ebook attachment, for example this is an extract of the exploded html:

Eliot, Charles W. (editor), Harvard Classics 22: Homer's Odyssey, v.1, 27 July 07
edited and uploaded by RWood
Download: http://www.mobileread.com/ebooks/mobi/4609  for   Harvard Classics XXII.prc

and 

Download: http://www.mobileread.com/ebooks/mobi/4610  for   Harvard Classics XXI.prc

As an example, try downloading "Bower, B. M - Cabin Fever" using these links:
.epub http://www.mobileread.com/ebooks/mobi/37998
.imp http://www.mobileread.com/ebooks/mobi/38000
.imp http://www.mobileread.com/ebooks/mobi/38001
.lrf http://www.mobileread.com/ebooks/mobi/38003
.prc http://www.mobileread.com/ebooks/mobi/38005

Now, if Alex would supply you with a downloadable listing of all the .prc/.mobi ebooks, then your app would just have to list it and allow the user to select which one to download (hopefully using filters/searches) and then fetch that attachment and place it into the iLiad's Books/Mobipocket folder in use.

Sounds simple doesn't it?

While I profess to have no inside knowledge of how the mobileread.com servers operate, I have tested this approach to downloading attachments before...

[hansel]

Thanks for the info!

http://www.mobileread.com/mobiguide looks like a great starting point. I converted it to epub with Calibre, unzipped it, converted the resulting html file to clean xml with htmltidy... The resulting file can be hacked with some xml-parsing scripts to extract the relevant information (and generate OPDS Atom XML). That is, unless such data is readily available at Mobileread...

Enough challenge for some rainy afternoons ;-)

[nrapallo]

Originally Posted by hansel

PS: a.f.a.i.k. the Iliad already can surf the internet (sort of) with minimo.

I had tried that but it was difficult getting started accessing the internet (probably was caused by using my 8GB CF card with wifi).

When I worked on impserve (just a local proxy server written in python by MR member ashkulz), I collected some convenient links into an internet "launch" sheet called index.html (see index-nrapallo.zip attachment). I tried accessing these links, but something is missing. The eBookwise devices had GREAT internal server/HTML support when working with HTTP GET and POST. The impserve python script ran on the user's local computer and served content to the device based on the mimetype of the links tapped in the hardware reader. Then impserve would "react" and fetch the web content, filter it, fix it for incompatibilities with the hardware devices' weaknesses and then "pass on the content" for the devices' internal mechanism to take over, be it downloading an ebook to the device or simply displaying the webpage previously fetched and fixed.

I'm sure the same approach could be used with the iLiad HOWEVER, there is no internal (server) support to accept a .prc/.mobi file for storage (or even perhaps a .pdf ebook) AFAIK. What we need is an internal server that can launch wired / wireless access and wait for files and/or webpages being served to it from the user's home computer that is running a proxy server like impserve. This new impserve (call it iliadserve or just ladserve?) could even be used to preprocess .epub locally and then send the resulting (exploded) files to the iLiad's internal webbrowser just like MR member mikelv has done with his Firefox add-on called EPUBReader.

BTW, another MR member Ret had previous to me used an alpha stage java proxy server/librarian that used a "sample.html" launch page to start surfing the internet by entering a keyword in an input box and having google retrieve it's links (attached retlibrarian.zip below). I tried it on the iLiad but there is no echoing of characters typed in the search box and it actually just retrieves the Google web search homepage where I have to type the search word again, this time getting the desired results. It would be nice to have the characters "echo" and use this link on an iLiad .html start page.

And prior to that I found C sourced code for software that was the basis of the inspirations for impserve and "surf the internet" on my LCD reader called LinReb. We (I) owe it all to this program for paving the way (see attachment below as well).

So, I think all the pieces are conceptually quite attainable, but nothing on the iLiad has been written to pull it all together. Above I have referenced a python script, html coding, java code and C source code. What an assortment of programming languages; all that is missing is the glue that combines them, ...., what? .... , did I hear you say Lua????

By the way, is there any better "internal" .html support that can replace (via an installed program) the use of minimo for surfing the internet ON the iLiad?

[hansel]

I'm not very interested in surfing the net with my Iliad (at the moment), more in downloading books with a simple gui. What I like about the Iliad is that I will not need a PC for that once there is some adequate software...

I'm pretty sure that the networking stuff on the Iliad works fine (as long as you don't access the cf while doing wifi). Wget is working fine, and can be used in scripts. Curl is o.k. too. The Lua binding for libcurl is included in my lua-iliad package. I did a small demo for Adam B. that downloads a file while showing progress.

It's easy to run a small webserver written in Lua on the Iliad. I did an experiment a while back (my first Lua for Iliad experiment) that could do 'server side scripting'. It's only a few hundred lines of code. It's not really nice code, but it worked

For now I'll focus on downloading books... soooo much to play, soooo little time

[nrapallo]

Originally Posted by hansel

I'm not very interested in surfing the net with my Iliad (at the moment), more in downloading books with a simple gui. What I like about the Iliad is that I will not need a PC for that once there is some adequate software...

A standalone app will surely do the trick, but while "surfing the net" the interface IS the html code fetched for the various web pages, like mobileread.com and when in the Mobi ebook forum, the user just has to click the link to the .prc/.mobi attachment BUT then the iLiad needs to know what to do with that mime-type. That's where the server side code, that is resident on the iLiad, could be helpful.

So, in the end, a really "simple" gui may just be any web page that has links to (your) mobipocket ebooks...

I'm pretty sure that the networking stuff on the Iliad works fine (as long as you don't access the cf while doing wifi). Wget is working fine, and can be used in scripts. Curl is o.k. too. The Lua binding for libcurl is included in my lua-iliad package. I did a small demo for Adam B. that downloads a file while showing progress.

I'll have to try that demo so thanks for the link! I did use wget to fetch Project Gutenberg .mobi ebooks before in a script, so I know it does work quite well.

It's easy to run a small webserver written in Lua on the Iliad. I did an experiment a while back (my first Lua for Iliad experiment) that could do 'server side scripting'. It's only a few hundred lines of code. It's not really nice code, but it worked

Oh, more code to play with. Thanks again!!!

For now I'll focus on downloading books... soooo much to play, soooo little time

and forget about any actual reading time...

[hansel]

Originally Posted by nrapallo

he way, is there any better "internal" .html support that can replace (via an installed program) the use of minimo for surfing the internet ON the iLiad?

I just read the 'xiliad' article again: dillo is another browser for the Iliad. Probably got it from http://www.mobileread.com/forums/showthread.php?t=8870...

Using a browser as gui and a web server as application was what I had I mind when toying with Xiliad. There was no scriptable gui then. It still is a viable approach with lots of possibilities...

Thanks for all the fresh ideas!

[nrapallo]

Originally Posted by hansel

That is, unless such data is readily available at Mobileread...

You will notice, in the Browse Latest Uploads section under the E-Books menu above in the blue bar, that there is a daily catalog of ALL mobileread.com ebooks in: Full List: HTML | TXT.

This catalog just links all ebooks to their description page, which contains the actual link to download the ebook attachment. For example, my above-noted links for the "Bower, B. M - Cabin Fever" ebooks posted by crutledge are listed as:

	<li><a href="http://www.mobileread.com/forums/showthread.php?t=60371">Bower, B. M: Bower, B. M. V1. 25 Oct 2009</a> <i>Mobipocket</i></li>
	<li><a href="http://www.mobileread.com/forums/showthread.php?t=60368">Bower, B. M: Bower, B. M. V1. 25 Oct 2009</a> <i>eBookwise</i></li>
	<li><a href="http://www.mobileread.com/forums/showthread.php?t=60370">Bower, B. M: Bower, B. M. V1. 25 Oct 2009</a> <i>Sony BBeB</i></li>
	<li><a href="http://www.mobileread.com/forums/showthread.php?t=60367">Bower, B. M: Bower, B. M. V1. 25 Oct 2009</a> <i>ePub</i></li>

If you filter by format name (Mobipocket for .mobi/.prc) then you can use that as a starting point. However, this approach requires a second (web) access to retrieve that description page so as to extract the actual download link. And then finally, you'll need to fetch that ebook file for the user and place it in the appropriate ebook folder.

This is crude but may offer some different thinking. This can be done only once, but will then ignore newly uploaded ebooks or daily to be current.

In the end, to really make your app/code shine, you will probably need a "feed" from mobileread.com with the pertinent data from Alex's server side data.

[nrapallo]

Originally Posted by hansel

It's easy to run a small webserver written in Lua on the Iliad. I did an experiment a while back (my first Lua for Iliad experiment) that could do 'server side scripting'. It's only a few hundred lines of code. It's not really nice code, but it worked

I didn't have any luck getting this to work with dillio 0.8.6 and Firefox 3.5 on my WinXP machine via wireless. It seems that it launches from the programs directory on my SD card, but then just returns to the ContentLister without giving my PC browser any access to http://192.168.X.XXX:8080 with my iliad's wireless ip address instead of those X's. Does it work with the latest firmware 2.12?

Originally Posted by hansel

I just read the 'xiliad' article again: dillo is another browser for the Iliad. Probably got it from http://www.mobileread.com/forums/showthread.php?t=8870...

Yes, thanks AGAIN for this link. It is an amazingly fast browser and quite feature rich. The xepdmgr dillio port looks nice as well but not all the same features (i.e. File Open) are supported in that port.

Using a browser as gui and a web server as application was what I had I mind when toying with Xiliad. There was no scriptable gui then. It still is a viable approach with lots of possibilities...

Thanks for all the fresh ideas!

Hopefully, whatever approach you decide to take in the end (standalone app or browser/proxy server) it will work well for the latest firmware and wireless.

p.s. Actually the ideas are rather old (been floating around my head for over a year), it's just me whose "fresh" to the iLiad scene! Oh, and as you make progress, I may have something to say about that too!

[hansel]

Originally Posted by nrapallo

I didn't have any luck getting this [Xiliad] to work...

I've repaired Xiliad, see: http://www.mobileread.com/forums/showthread.php?t=36799