webcomic2lrf, a brutal and simplistic commandline script

[acidzebra]

So I've settled into a comfortable routine with my Sony PRS-505, plugging it into my linux box at night which runs Calibre's feeds2lrf via a scheduled job so that in the morning I have my favorite news sources ready for consumption during my daily commute. I also enjoy a webcomic now and then (especially in the morning when my brain is still booting), so I hacked together a crude script that pulls in the latest of my favorite webcomics, zips them, then runs comic2lrf to create a nice LRF file.

To pull in the comics I use dosage, a program specifically designed for creating offline mirrors of various webcomics using 'recipes' which describe the URL and filename format the comic uses. Dosage is completely written in Python, which is why I chose to use it (there is other similar software out there).

Aside from Dosage, you'll need Calibre installed (duh). I run on linux but theoretically it should be possible to port this to other OSes. The script lives in a folder in my home directory called webcomic2lrf, and yes, the code is awkward and painful to behold but it works for me

#!/bin/bash
# First archive old comics and clean temp
mv -f ~/webcomic2lrf/Comics/ ~/webcomic2lrf/comicarchive/
rm -Rf ~/webcomic2lrf/tempcomic
mkdir ~/webcomic2lrf/tempcomic
# use the dosage binary 'mainline' to grab the comics
mainline ASofterWorld CatAndGirl CtrlAltDel DieselSweeties JerkCity JoyOfTech LittleGamers MakeWithTheFunny NuklearPower RealLife RedMeat QuestionableContent PvPonline TheOrderOfTheStick UserFriendly VGCats Wulffmorgenthaler xkcd Goats Dilbert SchlockMercenary UberSoft
# go to comic folder
cd ~/webcomic2lrf/Comics
# move everything to one folder as dosage creates subfolders for every comic
find . -name "*.*" -exec cp {} ~/webcomic2lrf/tempcomic \;
# go to tempcomic folder
cd ~/webcomic2lrf/tempcomic
# zip the comics
zip -m comic.zip *
# rename to cbz
mv -f comic.zip comic.cbz
# convert to lrf
comic2lrf -r -l -a "Webcomics" -t "Daily Webcomics" comic.cbz

Note that earlier I used the older "comiclrf" script which for some reason didn't like all these different images formats combined so I found it necessary to add one more step before zipping to convert everything to .jpg:

for img in *.*; do base=`basename $img`; convert "$img" "$base.jpg"; done

But since I switched to comic2lrf it all worked fine without it

Webcomics come in all shapes and sizes so some will be readable on your reader, some might not be. The 'recipes' that Dosage uses are sometimes out of date, but the format is easy enough to understand and adapt to any webcomic out there. Dosage also has some more advanced archiving and 'catching up' functionality but I haven't yet looked into it.

Anyhow, I hope this is of some use to the webcomic/prs-505/linux fans out there, all five of them

[ashadocat]

Make that six. Thanks this is really useful and saved me much sanity

[acidzebra]

Yay, six! Our numbers are growing!

I've got a slightly better version brewing which keeps track of when the reader was last attached and grabs all comics between now and the last connection but my complete lack of coding skills and procrastinating ways are stopping me from getting it to actually work.

[zelda_pinwheel]

hm. possibly seven. i love my comics. if i can figure this out i'd love to use it as well.

[acidzebra]

Hmmm, you would need a working python installation which is as simple as grabbing an installer from http://www.python.org/ , Calibre installed but that is a given I think, a commandline zip or rar program (to create the archive), and then try and run the dosage software.

Once you get that running the script should be easy to port to windows, these commands could be run from a batch script as easy as from a shell. It is the dosage program which is the crux - perhaps there is a way to grab offline copies of webcomics for windows already, in fact, I would bet on it.

[zelda_pinwheel]

mm hm. okay. i think i can manage most if not all of that. what exactly do you mean by a commandline zip or rar program ? can winrar be used via commandline ? can you recommend one, if not ?

also, when you say "batch file", do you mean copy that code into a txt and rename it *.bat ? or something more complicated ? in which case, what would it take to get you to make the batch file for me ?

[acidzebra]

Okay, for the webcomic grabbing maybe something like this 5 seconds of googling brought me:
http://www.softpedia.com/get/Others/Fun/Woofy.shtml
Haven't tested, haven't looked further :P

If you have winrar, there should already be two commandline programs installed in C:\Program Files\WinRAR - rar.exe and unrar.exe, you'll have to run them from a commandline to see what the possible switches are, I don't know.

As for batch files, yeah, it would be a little more involved. Linux uses a different structure for directories, you can see this in arguments used like "~/somefolder" which expands to "/home/acidzebra/somefolder", which in windows terms would be like c:\documents and settings\acidzebra.
I don't know if windows has a direct "move" (mv) command, but I'm sure that could be done with "copy" and "del" too. I think a "find" command also exists in windows 2000 and later versions but fecked if I know the syntax.

[zelda_pinwheel]

Originally Posted by acidzebra

Okay, for the webcomic grabbing maybe something like this 5 seconds of googling brought me:
http://www.softpedia.com/get/Others/Fun/Woofy.shtml
Haven't tested, haven't looked further :P

If you have winrar, there should already be two commandline programs installed in C:\Program Files\WinRAR - rar.exe and unrar.exe, you'll have to run them from a commandline to see what the possible switches are, I don't know.

ta. i'll look into it.

[gwynevans]

Dosage runs fine on Windows, although you need to install it and it's dependancies 'manually' (i.e. 'python setup.py install')

I'm using the following script conversion, although that itself uses the cygwin 'find.exe' in my D:\bin\ and A: is where I mount the Reader.

REM First archive old comics and clean temp
cd Z:\WebComics
cp -vr Comics/* ComicsArchive
rm -Rf Comics work
mkdir work

REM use the dosage binary 'mainline' to grab the comics
mainline ChasingTheSunset DieselSweeties JoyOfTech LittleGamers UserFriendly xkcd ComicsDotCom/getfuzzy Flipside KeenSpot/ElGoonishShive KeenSpot/ElGoonishShiveNP KeenSpot/JuvenileDiversion KeenSpot/MenageA3 KeenSpot/Striptease UClick/Doonesbury UClick/Foxtrot UClick/Garfield UClick/LibertyMeadows

REM go to comic folder
cd Comics
rem # move everything to one folder as dosage creates subfolders for every comic
D:\bin\find . -name "*.*" -exec "cp {} \WebComics\work" ;

rem # go to work folder
cd \WebComics\work
rem # zip the comics
zip -m comic.zip *
rem # rename to cbz
mv -f comic.zip comic.cbz
rem # convert to lrf
comic2lrf -r -l -a "Webcomics" -t "Daily Webcomics" comic.cbz
cd ..
copy work\comic.lrf A:\database\media\books\