View Single Post
Old 01-10-2012, 02:32 PM   #7
paulkon
Junior Member
paulkon began at the beginning.
 
Posts: 1
Karma: 10
Join Date: Jan 2012
Device: kindle touch
Found code for downloading blogspot archives

There's a blogspot blogger that I like to read. It goes back for 5 years, and so I wanted a way to download past years.

At first I thought that Calibre had a limit on the number of articles it could download from a blog, because it wasn't getting all of the articles available. But obviously it was nowhere near this limit.

Luckily someone wrote code to do this download. If you search the Custom Recipes thread for the "Universal Blogspot Downloader" it'll get you here:
https://www.mobileread.com/forums/sho...postcount=1785
which gives you recipe code, and some tweaking instructions. You can tweak it to specify the range of years and months that you want. As written, it downloads comments, too.

I decided to remove the comments, and so only show articles. My slightly revised version is attached. All I did was add a few new entries in the "remove_tags" line, based on reading the orginal HTML carefully. Big kudos to EnergyLens for the original.

Paul

BTW if you want to have articles sorted from oldest to newest, add this line to your recipe:
reverse_article_order = True
Attached Files
File Type: zip The Archdruid Report 2010_1001.zip (1.5 KB, 158 views)

Last edited by paulkon; 01-10-2012 at 02:44 PM. Reason: adding detail on sort order
paulkon is offline   Reply With Quote