Generate eBook from Bookmarks Archive

[vaniaspeedy]

Hello all. I regularly bookmark articles of interest to me, for later reference or easy access later. However, my reference folder is depended on those pages staying online and unchanged. I'd like to not only be able to download them for my records (which I already do), but also compile them into an eBook for easier reading and convenience. Here's the layout:

Abstract goal - turn some subset of my bookmarks into a logically structured ebook.

Specific Solution
1. Export bookmarks to html, then use a program such wget or httrack to download all of the the raw html files
2. Filter the html files - removing comments, ads, and extra fluff.
3. Using the html index from (1), with each link pointing to a cleaned html file, compile an ebook with a TOC, possibly recreating the folder structure of my bookmarks.

Main roadblocks:
- for (2) - I'm not sure how to do this. Any ideas for tools?
- for (3) - I need to find a way to preserve my folder structure.

I'd love some comments on the overall plan, which tools to use, and general feedback. Thanks!

[Toxaris]

What I would do, is upload those HTML files in Sigil and clean then up there. Layout/style is not to be retained probably anyway.
I would also ditch the folder structure, since it has no meaning in an ebook.
If you want to maintain it, I would advise creating it by hand. You can the information on the wiki or the site of jedisaber for that. It will be a good read anyway.

[dgillette.rm]

I would suggest starting with Instapaper or similar to help with the cleanup. I do not know what you mean by folders in a book sense. If you mean chapter and section, that should be easy in something like Sigil. If that is not what you mean, I agree with Toxaris that it is meaningless in an ebook.

[vaniaspeedy]

I've created an early alpha of a process that seems to work.

1. export bookmarks in html from firefox, leave only the links of interest (notepad++ is great)
2. clean using it the following regex expressions, using find and replace, replacing with empty blanks.
a) <[^>^A]+>
b) <A HREF="
c) " ADD_DATE="[0-9 ]+" LAST_MODIFIED="[0-9 ]+"
d) >[^<]+<\/A>
e) <H3 ADD_DATE="[0-9]+" LAST_MODIFIED="[0-9]+">.*

3. copy cleaned links to urls.txt
4. run this shell script:

#!/bin/sh

for url in `cat urls.txt `; do title=`curl $url 2>&1 | grep -i '<title>.*</title>' | sed -e 's/<[^>]*>//g'` &&  echo $url | mail -s "$title" YOUR_EMAIL@instapaper.com ; done

5. login to instapaper, download epub.

Limitations
Seems that instapaper only exports the last 20 unread articles, so I've been looking in to using a Calibre recipe that would download the newest 20, archive them, and grab the next 20. This loop could be run until I have a pile of epubs, which would be later glued together using some software.

Questions
1. Does anyone know of a prebuilt recipe that can do this?
2. Are there any programs that can automate the process of gluing together multiple eBooks, while also gluing together the TOC?

[DaleDe]

For recipe questions I would suggest you post in the recipe subforum of the Calibre forum.