Introducing Calkin, a simple one-way fetcher server <->kindle (e.g. calibre recipes)

[forensic]

Regards!

Introducing Calkin, a simple Calibre <-> Kindle News feed fetcher and general file deliverer

From the README:
"Features" (or: what is this?):
- First all all: No binary blobs, very small, you can check everything for yourself. You don't have to trust me.
- Calkin relies on Calibre for the fetching of news feeds (rss, or more broadly: recipes)
- "one-way" sync any file (e.g. binaries, pdf, epub, mobi etc.) from a specified location to your kindle (e.g. parts of you library) (it uses md5checksums to reduce bandwitdth)
- Once setup, you just have to have your wifi on and the server (or whatever you are running) on, and you will be provided with fresh news in the background
- This was bourne out of lack of a better alternative (e.g. paid subscription and dependency on even more services and servers), with this just the kindle, your box (and if you want news, the site from which you want to fetch them) have to be online (or rather:connected).
- The Process:
Server: Cron calls fetch.sh -> get's list of recipes, calls calibre, which fetches them. Time is tracked. A file for the Kindle <-> Server communation is generated.
Kindle: Cron on Kindle calls calkin-kindle-sync -> calkin-kindle-sync checks wheather connection to server is possible -> Generates list of files already fetched to (hidden?) location and md5sum of files -> Uploads list to server -> Calls script on server, server generates md5sum of every file contained in defined folder (on server) and checks them against the ones in the kindle list -> Provides download links for kindle -> kindle replaces changed files with new ones, updates home screen.

ONLY Tested on Ubuntu 12.04 TLS (bash version 4.2.45(1)) + Kindle K3W + apache2 + php 4.9.4

Requirements:
-Calibre for your distro
-Newsfeed recipes for Calibre
-VServer, HomeServer, etc., at least Apache2 (sudo apt-get install tasksel,sudo tasksel Web-Server) (or virtual)
-or just a webspace with file access (than you have to upload the files (e.g. via ftp or the like); tweak as you like)
-corresponding port forwarding to Server and Firewall exceptions

-Kindleside: Jailbreak, Usb Network (?)

Known limitations:
-Currently weird filenames (with whitespacesetc.) don't work

INSTALL:
Please read the provided README

Questions? Suggestions?

[knc1]

You are doing all of your file operations directly in flash !
Not a great idea for several reasons.

Stage your file operations -

1) Using a size limited, tmpfs backed, file system -
Do all your file operations that you can in this stage.

2) When a file is completely ready -
Transfer it to flash with a single "mv ... ..." command.

I do not see any handling of IFS in the script.
That may well be part of your filename handling problems.

Another thing to consider about the filename handling -
The file will end up on a FAT32 filesystem, with all that implies.
PLUS
You might want to view the options that file system is mounted with in the Kindle, there may also be characterset problems.

[NiLuJe]

Have you looked into rsync? It's bundled with USBNetwork, and it pretty much does what you want (ie: one or two way directory sync) without extra round-trips, and a simple rsync server instead of a LAMP setup.

(I might be biased in favor of rsync as a Gentoo user, but, honestly, it rocks! )

[knc1]

Originally Posted by NiLuJe

Have you looked into rsync? It's bundled with USBNetwork, and it pretty much does what you want (ie: one or two way directory sync) without extra round-trips, and a simple rsync server instead of a LAMP setup.

(I might be biased in favor of rsync as a Gentoo user, but, honestly, it rocks! )

You don't even need the rsync server if you have ssh-pubkey access to the remote file system.