Wikipedia conversion

[fogus]

Hello all,

I would like to request info from anyone on their conversion of Wikipedia pages to read on their Sony reader (I'm using the 505).

Right now I am using the book conversion utility and saving in open document format. Unfortunately I cannot find a way to remove all of the footnotes (which makes the numbers get highlighted automatically in the text).

Does anyone have a good conversion method? I'd like the titles to be larger, extra lines between paragraphs and NO footnotes whatsoever. I can't find a way to remove footnotes in open office or in calibre.

Cheers,

[pilotbob]

I do plan to give it a whack using the Wikipedia DVD. I just haven't got to it yet.

BOb

[kiklop74]

Try this recipe. It is able to scrap any wiki page. You just need to setup manually list of articles for different topics. Like this:

    ...
    INDEX = 'http://en.wikipedia.org/'
    ...

    def parse_index(self):
        articles = []
        articles.append({
                         'title':'Tree',
                         'date':'',
                         'url':self.INDEX + 'wiki/Tree',
                         'description':'Tree'
                        })
        articles.append({
                         'title':'Tiger',
                         'date':'',
                         'url':self.INDEX + 'wiki/Tiger',
                         'description':'Tiger'
                        })
         
        return [('Articles', articles)]

than just run the recipe and you will get the ebook without images.

This leaves space for improvement but it is something to start working on.

[fogus]

I see. That's really great!

Is there a way to grab multiple articles and have each article become a chapter (i.e., an item in the table of contents) inside one book? That'd be nice.

[kiklop74]

That is exactly how this recipe works

[bobbyperu]

Thanks, nice work.

[fogus]

http://pastebin.com/f3f43c6f9 (my attempt) gives:

---------------------------
ERROR: Invalid input
---------------------------
<p>Could not create recipe. Error:<br>unexpected indent (recipe3.py, line 57)
---------------------------
&OK
---------------------------

Why is there an unexpected indent on line 57?

[fogus]

It turns out calibre gives different line numbers from a command line run:

U:\...Scripts>python wikipedia_two.py
File "wikipedia_two.py", line 53
def parse_index(self):
^
IndentationError: unexpected indent

After seeing that we were able to fix the error. (my tripple quoted section above line 53 was not indented.

Could the line number reporting be fixed?

Cheers,

[kovidgoyal]

the reason for that is that the GUI adds a few lines to the top of the recipe to make sure it runs, the command line doesn't

[fogus]

Is that a static four lines? (then I could subtract it off and find the real error)

[kovidgoyal]

Yeah it is don't recall if it's exactly four or some other number but that is easy to test by deliberately introducing an error

[fogus]

Based on the difference observed above I believe that it is strictly four lines (57-53=4). Good info to know.