Dropbox client, Seafile client

[trap000d]

Please welcome to test my little projects:

https://github.com/trap000d/seafile-kindle-client
https://github.com/trap000d/dropbox-kindle-client

They both are made as KUAL extensions and work similarly so installation instructions are similar too. In short words: download, unpack to extensions folder, edit config, start KUAL, use.

I refrained from implementing daemon mode for several reasons:
1. Almost all the time I keep my Kindles in airplane mode, therefore these scripts when running in background will only eat resources.
2. Frequent notifications are distracting, but without notifications it's not possible to get information about synchronization process.
3. If you remove or replace open document, the result is unpredictable.

Longer description
Seafile client:

Spoiler:

# seafile-kindle-client
Simple client for synchronization between Kindle and Seafile server. Upload works only for particular direectory.

### Installation

- Your kindle must be jailbroken
- Install python for kindle https://www.mobileread.com/forums/sho...d.php?t=225030
- Copy the contents of KUAL/seafile directory into /mnt/us/extensions/seafile
- Copy seafile.cfg.example to seafile.cfg, set there proper server address, library name, your login and password (as well as path to local directory):
```
[server]
url = https://seafile.example.com
library = MyBooks
user = user@mail.server
password = Password
; SSL certificate verify options: True, False or path to self-signed crt file
cert = False

[kindle]
local = /mnt/us/documents/Seafile
upload = /MyKindle_1
; screen dimensions in chars: 68x60 for PW3/KV, 48x42 for PW2
width = 68
height = 60
```
Run it via KUAL menu:
- KUAL -> Seafile Sync -> Synchronize
or
- KUAL -> Seafile Sync -> Push to server

At the first run script will obtain an authentication token at Seafile server and remove your login and password entries from configuration file (for security reasons).

### Known Issues/Bugs/Limitations
- Synchronization (download) works pretty well for 100-200 files in 3 sub-dir levels with overall size 1 Gb. For testing a bigger volumes I'll have to re-solder Kindle storage chip and hack firmware.
- One'n'half-way synchronization (only newly created local files are uploaded to server). As ID of file is generated on the server, there is no reliable way to determine if file is changed locally by it's ID. File timestamp doesn't look good too as kindle clock might reset after cold restart.
- There is an option for uploading of the particular directory contents (useful e.g. for notes synchronization). As all files in that directory have to be uploaded to the server you should be careful: it could take much time.
- Directory for uploads must exist on the server. You have to create it there (e.g. via web interface or with desktop seafile client) and perform synchronization (download) at least once before upload.
- Upload directory must be a sub-folder in directory tree, e.g. /mnt/us/documents/Seafile/MyKindle_1. In config it should be defined as relative path to the base directory
- File ID is a hash of file therefore all exact copies of particular file will have the same ID (though different names). Because the client idea is based on ID comparison, then if you make a copy of existing file at the server, client will download only the first file.
- Just rudimentary checks of internet/WiFi availability/file operations
- Hidden files/folders are not synchronized as well as bookmarks/statistics (*.sdr). It's because client has keeping actual state in hidden files ".hash", also some FUSE FS garbage often present as .fuse_hiddenXXXXXX.

Dropbox client:

Spoiler:

# dropbox-kindle-client
Dropbox client for Amazon Kindle
Based on Seafile client https://github.com/trap000d/seafile-kindle-client

### Installation

- Your kindle must be jailbroken
- Install python for kindle https://www.mobileread.com/forums/sho...d.php?t=225030
- Copy/unpack the contents of KUAL/dropbox directory to /mnt/us/extensions/dropbox
- Copy dropbox.cfg.example to dropbox.cfg, set Dropbox folder to sync, as well as your Dropbox token (see howto generate token here: https://www.dropbox.com/developers/r...ng-up-your-app)

```
[server]
; Dropbox folder to synchronize
library = MyBooks
; Dropbox API Token
token = s7ergngt3y3fhdsnvjdnvnjfbfywfgywgcsdsdbsd

[kindle]
; Local Kindle directory to synchronize
local = /mnt/us/documents/Dropbox
; contents of /mnt/us/documents/Dropbox/MyKindle_1 ("local"+"upload") will be forced to upload
upload = /MyKindle_1
; screen dimensions in chars: 68x60 for PW3/KV, 48x42 for PW2
width = 68
height = 60
```

### Run

Via KUAL menu:
- KUAL -> Dropbox Sync -> Synchronize
or
- KUAL -> Dropbox Sync -> Push to server
In this case all the contents of directory 'local+upload' will be forced to upload into Dropbox (useful e.g. for notes synchronization).

Via command line:

```/mnt/us/extensions/dropbox/bin/dbcli.py ``` for sync
```/mnt/us/extensions/dropbox/bin/dbcli.py push ``` for upload

### Known Issues/Bugs/Limitations
- One'n'half-way synchronization (only newly created local files are uploaded to server). As ID of file is generated on the server, there is no reliable way to determine if file is changed locally by it's ID. File timestamp doesn't look g
ood too as kindle clock might reset after cold restart.
- There is an option for uploading of the particular directory contents (useful e.g. for notes synchronization). As all files in that directory have to be uploaded to the server you should be careful: it could take much time.
- Directory for uploads must exist on the server. You have to create it there (e.g. via web interface or with official Dropbox client) and perform synchronization (download) at least once before upload.
- Upload directory must be a sub-folder in directory tree, e.g. /mnt/us/documents/Dropbox/MyKindle_1. In config it should be defined as relative path to the base directory
- Just rudimentary checks of internet/WiFi availability/file operations
- Hidden files/folders are not synchronized as well as bookmarks/statistics (*.sdr). It's because client has keeping actual state in hidden files ".hash", also some FUSE FS garbage often present as .fuse_hiddenXXXXXX.

What's new:
0.1.2

• More intelligent network errors handling
• Automatic detection of screen size: you can remove 'height' and 'width' parameters from config

0.1.3

• Automatic wireless state detection, activation and deactivation.
• bugs/issues/UI fixes

ToDo:

override problems with non-Unicode 'bad' characters in file names:

• legacy 8859-1 (Latin-1) symbols are still in use in Germany (and probably other countries), but are not interpreted properly by official Dropbox client
• Other symbols not compatible with Kindle FAT file system, e.g. <,>,?,#,:

Download links for current versions:

Seafile client:
https://github.com/trap000d/seafile-.../v0.1.3.tar.gz

Dropbox client:
https://github.com/trap000d/dropbox-.../v0.1.3.tar.gz

Find new releases here:
https://github.com/trap000d/seafile-...lient/releases
https://github.com/trap000d/dropbox-...lient/releases

[encol]

nice work, thanks for sharing

[scissors]

Hi
I'd appreciate some "help for a dummy" please.
I copied the dropbox folder to G:\extensions\

when I open KUAL dropbox sync is in the list.
I renamed dropbox.cfg to .old
I renamed .example to .cfg
I went to my dropbox and (this is where things went wrong i think) I created a new app, which i just called it " my kindle"
I copied the token into the dropbox.cfg

I ran the sync with log and unsurprisingly it didn't work.

The log created in my documents folder is as such

Traceback (most recent call last):
File "/mnt/us/extensions/dropbox/bin/dbcli.py", line 8, in <module>
from requests.packages.urllib3.exceptions import SubjectAltNameWarning
ImportError: cannot import name SubjectAltNameWarning


As a bog standard user i could do with guidance. I can't even see how you let dropbox know it is my dropbox to sync to (unless dropbox knows from the token)

Thanks in advance

[trap000d]

Hi Scissors,

You did all things absolutely right. I'm just not sure if you have installed Python properly.

I assume you've got Kindle 3G device (as it shown in your profile info). Unfortunately I have no one on hand to test and debug (I have only 3 exemplars at home: PW1, PW2 and PW3 resp.).

That particular line which has caused an error, actually useless for Dropbox (I've forgot to remove some Seafile client code). Please give a try to a new version (archive attached to this message) and send me log.

P.S. As for Dropbox API token. It's a standard authentication and authorization scheme for Dropbox clients (a good alternative to login/password pair). Server recognizes applications of particular user and their credentials by single token. A very brief explanation how it works you may find here: https://www.dropbox.com/developers/r...ce/oauth-guide

Originally Posted by scissors

Hi
I'd appreciate some "help for a dummy" please.
I copied the dropbox folder to G:\extensions\

when I open KUAL dropbox sync is in the list.
I renamed dropbox.cfg to .old
I renamed .example to .cfg
I went to my dropbox and (this is where things went wrong i think) I created a new app, which i just called it " my kindle"
I copied the token into the dropbox.cfg

I ran the sync with log and unsurprisingly it didn't work.

The log created in my documents folder is as such

Traceback (most recent call last):
File "/mnt/us/extensions/dropbox/bin/dbcli.py", line 8, in <module>
from requests.packages.urllib3.exceptions import SubjectAltNameWarning
ImportError: cannot import name SubjectAltNameWarning


As a bog standard user i could do with guidance. I can't even see how you let dropbox know it is my dropbox to sync to (unless dropbox knows from the token)

Thanks in advance

[notimp]

Thank you very much for your work and this great extension. Sync is faster than with other solutions running on top of Amazons Whispersync in the past.

I only found one shortcoming so far and that is, that files don't get downloaded correctly if there is an unexpected non-standard character (like an umlaut) in the filename.

But I was able to easily fix this with a shell script on my Mac (thats used to move files to the Dropbox folder).

Also, I managed to get an entire soft-hyphenation and conversion workflow set up using only the command line - so the azw3 file that now ends up in my sync folder already is hyphenated once it reaches the Kindle, after running one simple script.

My (macOS) script (save as a .sh file), if some of you wan't to set up something similar:

cd ~/Downloads

mv "$(ls -t | grep '\.epub$' |head -1)" ~/Dropbox/

sleep 0.5

cd ~/Dropbox

cp "$(ls -t | grep '\.epub$' |head -1)" ~/Dropbox/Books/

cd ~/Dropbox/Books

find . -name '*.epub' -exec /Applications/calibre.app/Contents/console.app/Contents/MacOS/calibre-debug -e ~/Documents/kindle-hyphens/main.py {} {} \;

sleep 0.5

/Applications/calibre.app/Contents/console.app/Contents/MacOS/ebook-convert "$(ls -t | grep '\.epub$' |head -1 | grep -Po '.*(?=\.)' | sed 's/(\(.*\))//' | sed -e 's/[^A-Za-z0-9._-]/ /g').azw3" --share-not-sync --change-justification justify

rm "$(ls -t | grep '\.epub$' |head -1)"

sleep 1

killall Terminal

This script uses calibre (you also have to have the Modify ePub plugin (https://www.mobileread.com/forums/sh...d.php?t=154371) installed) and kindle-hyphens (https://github.com/dmalinovsky/kindle-hyphens).

To use kindle-hyphens you must have Calibre (with the Modify ePub plugin), Python and lxml installed.

To install lxml read the kindle-hyphens readme, or look at this thread: http://stackoverflow.com/questions/1...-mac-os-x-10-9

What the script above does:

Takes the most recent epub in ~/Downloads, moves it to ~/Dropbox and copies it to ~/Dropbox/Books , lets kindle-hyphens put soft hyphens in the epub in ~/Dropbox/Books, uses calibre to convert it to an .azw3 (and some | grep | sed magic to remove all special characters and brackets (including the stuff in the brackets) from the filename, then deletes the .epub in ~/Dropbox/Books (.azw3 remains there of course), but leaves the one in ~/Dropbox for archival purposes.

Have fun setting up a similar workflow for yourselves.

n.

[scissors]

Originally Posted by scissors

Hi
I'd appreciate some "help for a dummy" please.
I copied the dropbox folder to G:\extensions\

when I open KUAL dropbox sync is in the list.
I renamed dropbox.cfg to .old
I renamed .example to .cfg
I went to my dropbox and (this is where things went wrong i think) I created a new app, which i just called it " my kindle"
I copied the token into the dropbox.cfg

I ran the sync with log and unsurprisingly it didn't work.

The log created in my documents folder is as such

Traceback (most recent call last):
File "/mnt/us/extensions/dropbox/bin/dbcli.py", line 8, in <module>
from requests.packages.urllib3.exceptions import SubjectAltNameWarning
ImportError: cannot import name SubjectAltNameWarning


As a bog standard user i could do with guidance. I can't even see how you let dropbox know it is my dropbox to sync to (unless dropbox knows from the token)

Thanks in advance

I'm assuming python is correct as i have the screen saver hack working.
I copied over your new file and set the screen to 40 x 50 (as in the comment)

now i get this.

Spoiler:

eips: pixel_in_range> (24, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (48, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (60, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (72, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (84, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (96, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (108, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (120, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (132, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (144, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (156, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (168, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (180, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (192, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (204, 980) pixel not in range (0..608, 0..800)
eips: pixel_in_range> (24, 980) pixel not in range (0..608, 0..800)
Traceback (most recent call last):
File "/mnt/us/extensions/dropbox/bin/dbcli.py", line 388, in <module>
rc = db_authping()
File "/mnt/us/extensions/dropbox/bin/dbcli.py", line 93, in db_authping
jResp = r.json()
File "/mnt/us/python/lib/python2.7/site-packages/requests/models.py", line 805, in json
return complexjson.loads(self.text, **kwargs)
File "/mnt/us/python/lib/python2.7/site-packages/simplejson/__init__.py", line 516, in loads
return _default_decoder.decode(s)
File "/mnt/us/python/lib/python2.7/site-packages/simplejson/decoder.py", line 370, in decode
obj, end = self.raw_decode(s)
File "/mnt/us/python/lib/python2.7/site-packages/simplejson/decoder.py", line 400, in raw_decode
return self.scan_once(s, idx=_w(s, idx).end())
simplejson.scanner.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

hopefully that means something to you

[trap000d]

Hi Notimp,

Thank you.

I tested scripts for compatibility with unicode (files with Cyrillic and Chinese characters in names). If you send me an example of name which caused trouble, I'll find where is the problem: in Dropbox, in my scripts or somewhere else.

[trap000d]

Hi Scissors,

Answered to PM

[notimp]

Originally Posted by trap000d

Hi Notimp,

Thank you.

I tested scripts for compatibility with unicode (files with Cyrillic and Chinese characters in names). If you send me an example of name which caused trouble, I'll find where is the problem: in Dropbox, in my scripts or somewhere else.

Here is one of the files that innitially caused me some issues: https://www.load.to/WORsnnsFiO/Renzi...pload.biz.epub

Mirror:
http://www.filefactory.com/file/3snt...pload.biz.epub

The file seems to download fine, but then on the device turns out to be a 8kb file with a dropbox api error encoded into it (= error at dropbox' end?) and not the initial epub - if I replace the umlauts (here are some you could test against - öüäÖÜÄß ) with spaces, it downloads (/syncs) fine.

I'm on a PW3, and the extension works fine for me, btw.

edit: And just as a small aside, if you are using Linux and looking at my script - the calibre command line commands should be global on Linux - if you delete the /Applications/calibre.app/Contents/console.app/Contents/MacOS/ parts in the code above, the lines should start working for you - at least thats what Kovid states in the Calibre documentation.

[trap000d]

Hi Notimp,

For me it looks like a problem at Dropbox side. After I've downloaded your file and placed it into Dropbox folder, it has been immediately renamed. Now I've got 2 files here: file with original name ( but zero size), and a new one with name "[name looks exactly like original but I'm not sure](Unicode Encoding Conflict).epub"

They explained this behaviour here

https://ask.libreoffice.org/en/quest...h-libreoffice/

Originally Posted by notimp

Here is one of the files that innitially caused me some issues: https://www.load.to/WORsnnsFiO/Renzi...pload.biz.epub

Mirror:
http://www.filefactory.com/file/3snt...pload.biz.epub

The file seems to download fine, but then on the device turns out to be a 8kb file with a dropbox api error encoded into it (= error at dropbox' end?) and not the initial epub - if I replace the umlauts (here are some you could test against - öüäÖÜÄß ) with spaces, it downloads (/syncs) fine.

I'm on a PW3, and the extension works fine for me, btw.

edit: And just as a small aside, if you are using Linux and looking at my script - the calibre command line commands should be global on Linux - if you delete the /Applications/calibre.app/Contents/console.app/Contents/MacOS/ parts in the code above, the lines should start working for you - at least thats what Kovid states in the Calibre documentation.

[scissors]

Originally Posted by trap000d

Hi Scissors,

Answered to PM

Hi There,
tried both the .py and complete package.
same problem.

Gonna give up, too much other stuff going on. Might try again in the new year when stuff quieter.

Best Regards

[trap000d]

Hi Scissors,

Sorry I've not managed to find the root of problem on your device. Hope you'll have some free time in new year. Unfortunately I can't find any similar device around for testing.

Regards,

Originally Posted by scissors

Hi There,
tried both the .py and complete package.
same problem.

Gonna give up, too much other stuff going on. Might try again in the new year when stuff quieter.

Best Regards

[notimp]

Not sure if possible - but, could you have your script activate the Kindles Wifi (- Kindle then auto connects) on start, and deactivate it again once sync is finished? It would be neat to have that functionality. Those extra steps become quite tedious after a while..

[knc1]

Originally Posted by notimp

Not sure if possible - but, could you have your script activate the Kindles Wifi (- Kindle then auto connects) on start, and deactivate it again once sync is finished? It would be neat to have that functionality. Those extra steps become quite tedious after a while..

There are lipc commands for those purposes that probably could be included in the script.

Give it a try and let us know.

[notimp]

Commands are:

disable WiFi: lipc-set-prop com.lab126.cmd wirelessEnable 0
enable WiFi: lipc-set-prop com.lab126.cmd wirelessEnable 1

src: https://wiki.mobileread.com/wiki/Kind..._like_the_menu

I'll see if I can get it hacked together.