Wikipedia on Sony Reader

[yekim54]

Originally Posted by ahi

Warning: Don't ever try to actually use an *external* table of contents that has over a hundred thousand entries.

Thanks for playing guinea pig and checking that out for us!

[Fabiegail]

Hi
i'm a complete newbie at this ebook reader thing (got mine three days ago), so I'm still trying to figure things out. Wikipedia dumps are XML, and we're trying to compress and index it in a way that a reader can use, is it right?
I'm not sure why the thing that works for Irex ( http://code.google.com/p/pyoffwiki/ ) doesn't for the Sony reader. Is it a format problem?
Sorry to ask rookie questions, but I'd like to understand the problem better, even though I know I can't help.

[erio]

Fabiegail, you're newbie? I don't even have a reader and when i read all you its like you speak in chinese... but i'd also wish i could help even if i also know i can't... sigh

[Fabiegail]

Well, I did my homework...
I'm getting frustrated, because I feel the solution can't be *that* complicated, but I don't have the skills to devise it. Especially considering that The Wikireader has already implemented all the tools to parse, compress and index the WP dumps, so that much is done:
http://github.com/wikireader/wikireader
So what remains to be done is the Sony Reader part (Linux + JScript) http://www.reghardware.co.uk/2009/08...k_sony_reader/

I so wish I could program in Python!

[kartu]

Originally Posted by gonzule

How on earth am i going to put 1.3GB of data on my reader? I can get a cheap 2 or 4 GB sandisk memory stick duo (or SD card) and store the files there, the thing is that it will be more than 600,000 files to store and that the reader will have to read an manage. Is the reader even able to handle such an ammount of files??? will the baterry drain just from trying to list the files on the book collection?

With this:
https://www.mobileread.com/forums/showthread.php?t=64510

you can disable file scanning. In this case you won't be able to open files directly from the card, but will get an option to copy file you want to the internal memory and then open it.

[chrisperry]

So... Any advances in running wiki on Sonyreader?
As far as I know, it can be downloaded in xml:
http://download.wikimedia.org/enwiki...ticles.xml.bz2
So, the possible solution would be:
a) create a hack so that Sony Reader can open this xml.
b) find a way to convert this file to readable format.

[kartu]

chrisperry
5.7Gb. Is there any solution on stripping it down to a reasonable size?

[Duglum]

I downloaded a dump of a smaller language (piwiki, 1.7MB). To make matters worse, it looks like all the data is only available in one single huge XML-file. Notepad++ crashes after 10 minutes while trying to open it. I don't think it's viable to put such a monster on a Sony Reader.

If there was a single file for every wikipedia page it would maybe be possible to compress the site and decompress it when the user wants to read it. There is a ZIP-implementation in JS for example: http://jszip.stuartk.co.uk/

[chrisperry]

Originally Posted by kartu

chrisperry
5.7Gb. Is there any solution on stripping it down to a reasonable size?

No, I don't think so. (What is a reasonable size, anyway?)

OK, you can use a better compression method (7z for example), but you will get only few extra %.(And opening the file would be longer and complicated.)

Or you can remove some articles. But which articles to remove and which to leave? This must be done manually and there are millions of them. (If you are interested, look here: http://meta.wikimedia.org/wiki/Wikipedia_on_CD/DVD)

But this is not what I want. First, there must be a WAY to to open wiki on PRS reader or to convert this huge xml to some readable format. Some kind of framework. Than everybody should decide if he wants some basic wiki with few most important articles (few MBs), the complete one (5,7GB... well, if you are really into this, 8GB memory cards aren't THAT expensive).

[CapnKirk]

So... there's been quite a bit of technical discussion of the problems and issues. Is anyone successfully and practically reading Wikipedia on their Sony?

I have a prs-600 with a 32GB sd card, so I've got a fair bit of room.

[Valwit]

why not creatinmg several books for different letters, like it used to be with encyclopedias before the computers? I mean a book for AB, anoder for CD and so on. this way they can be smaller then just one large file

[gregcd]

I take a "just in time" approach to getting Wikipedia on Sony Reader - it's possible to get any web page on to a micro SD or m2 card, using a mobile phone and web2fb2.net & opera mini. It's a bit of a hassle card swapping, but it only takes a half a minute including conversion.

First open the Wikipedia page in opera mini on your phone. Go to bookmark it, but instead then copy the page URL. Paste this in to the web2fb2 page for conversion to epub.

You'll need an SD or Duo adapter of course to put the card in your reader.

Web2fb2 outputs to a epub file with images and formatting included.

[Ayle]

Originally Posted by Valwit

why not creatinmg several books for different letters, like it used to be with encyclopedias before the computers? I mean a book for AB, anoder for CD and so on. this way they can be smaller then just one large file

Me likey.


Or maybe by area of interest, dunno.