FreeDownload hack

[dsmid]

FreeDownload

EDIT: Uninstall this hack before upgrading the firmware or the upgrade would fail. After upgrade you can install it again.

This hack is a workaround for Kindle browser extension/mime type restrictions.
The browser permits downloading only .prc, .mobi, .azw and .azw1 files. Yet it may be handy to download other file types as well, e.g. .pdf, .epub, .rtf, .zip, .cbz ...

FreeDownload doesn't modify the browser itself, instead it turns on logging of some interesting info the browser is able to produce.
Then it processes the info generated by the browser, i.e. the complete URIs the browser is about to download. If the URI has one of the specified extensions, FreeDownload passes it to curl for download.
At the same time it tries to inform user about downloading at the bottom of the screen. Despite the nagging message box being displayed, your file WILL be downloaded. Works over Wi-Fi only ATM.

FreeDownload's behaviour can be controlled by modifying freedownload.conf located in [USER STORE]/freedownload/ directory.

Default contents of freedownload.conf :

# List extensions you want to have downloaded by FreeDownload
# Make sure you don't specify permitted extensions like azw or mobi, your files would be downloaded twice
ALLOWED_EXTENSIONS='pdf epub rtf zip cbz mp3'

# Where should FreeDownload put your files ? The directory will be created if not present
# If / is missing at the beginning of the path, "/mnt/base-us/" (AKA user store) is prepended
DOWNLOAD_DIR=documents/Downloaded

# Where should FreeDownload put files with 'mp3' extension ? The directory will be created if not present
# You can specify separate download directory for any extension in a similar way
DOWNLOAD_DIR_mp3=music

# Where should FreeDownload put files with 'aax' extension
DOWNLOAD_DIR_aax=audible

# Resume previous download if a local file with the same name exists
# Target files are overwritten if this option is disabled
RESUME_DOWNLOADS=yes

# Unzip any file with .zip extension into its own subdirectory after download
# ZIP archives containing illegal (non-utf8) file names won't be unzipped
#AUTO_UNZIP=yes

# Log all downloads and errors to freedownload/freedownload.log
#LOG=yes

Changelog:
0.4
- Tried to fix CPU overload occuring in some situations

0.3
- Fixed serious issue causing damaged user store (redesigned, autosuspend the service when USB connection is detected)
- Curl now tries to use browser's cookies file
- Reporting now works in landscape mode
- Progress is displayed when downloading
- ZIP archives containing filenames with FAT32-illegal and non-ascii characters are not autounzipped
- Separate download directory can be defined for each extension
- No need to specify full paths for download directories
- In freedownload.conf Windows line endings are converted to UNIX line endings on start
- Added option that controls resuming/overwriting downloads

0.2
- Used curl instead of wget to support https (DropBox Public files download fixed)
- Added option AUTO_UNZIP

0.1
- Initial release

[xlendi]

Dsmid, that's great! Good job!

[dsmid]

New version 0.2 :

• Used curl instead of wget to support https (DropBox Public files download fixed)
• Added option AUTO_UNZIP

[stasenko]

Thank you for your work. Can you make different download directory for different file extension? For example, '/mnt/us/documents' - 'mobi', '/mnt/us/music' - 'mp3', etc.

[ewandeep]

I wonder if it works with the kindle dx graphite.

[Wild Rat]

Warning! if you download and unzip the archive containing the files whose names contain non Latin characters, then:
1) Unzip the files will have incorrect names
2) Kindle will not see them
3) some files will not delete!

[xlendi]

I had also problems to uninstal the first version (0.1).

[dsmid]

Please don't install this hack (versions 0.1 and 0.2), it will disable copying files over USB storage and as a result installing any hacks or updates (UPDATE ERROR: 3).
I didn't find out because I don't use the USB connection at all, I use sftp over Wi-Fi instead. Sorry for any inconvenience.

If you already did install it, here's how to get rid of it and make the user store usable again:

1. Delete the whole directory 'freedownload' in user store
2. Restart your Kindle
3. Copy appropriate update_freedownload_0.*_k3*_uninstall.bin to user store then go to Settings and select 'Update your Kindle' in the menu.
4. Restart the Kindle again
5. Now your Kindle's user store should be OK again, you can check the filesystem for errors from your PC to make sure everything is all right

[dsmid]

Originally Posted by Wild Rat

Warning! if you download and unzip the archive containing the files whose names contain non Latin characters, then:
1) Unzip the files will have incorrect names
2) Kindle will not see them
3) some files will not delete!

The problem is, that sometimes the ZIP archive contains filenames that use some obscure Windows language-specific encoding. Your Kindle isn't able to find out what encoding Windows uses in your country.
If the filenames are encoded in UTF-8 (the only reasonable standard today), everything's OK.

[dsmid]

Originally Posted by stasenko

Thank you for your work. Can you make different download directory for different file extension? For example, '/mnt/us/documents' - 'mobi', '/mnt/us/music' - 'mp3', etc.

When I properly fix the issues with user store access, I'll implement this feature. Stay tuned.

[otichy]

Maybe the freedownloader should run from somewhere else than /mnt/us ? Or it could stay stored there but run from a temporary location? I don't think the non-unicode characters are the only problem.

[dsmid]

Originally Posted by otichy

Maybe the freedownloader should run from somewhere else than /mnt/us ?

Yes, it shouldn't open any files on the user store (including its own executable) and this is quite tricky because downloading file usually means opening a file in the user store for writing. Any open files would prevent unmounting the user store on Kindle side when the USB cable is connected to host.
Another option is to stop the service whenever the USB connection is detected - I suppose usbnetwork's ssh daemon is working like that.

[otichy]

Stopping the service would be the best solution. Otherwise, it should run from somewhere else and it might download the file to some temp location and once it's finished, copy it over to the /mnt/us. The only problem would be if somebody started to download stuff with the hack while having the USB plugged in.

[dsmid]

I agree. Stopping the service would be the safest solution.
I had the idea with temp location for downloads before but I was afraid the tmpfs partition is not big enough to hold large downloads and I don't support the idea of dowloading anything to the system partition (running out of disk space there doesn't sound safe).

[Tiersten]

Running out of space on the user store makes the Kindle go nuts so filling the other partitions is a really bad idea!