DB Auto-Backup Tool for Kobo

[Lucas Malor]

Since I got the database corrupted very much times, I decided to do this little auto-backup tool.

It simply backups the db every 20 minutes, or when the USB cable is attached, in a hidden place. If the database is corrupted, LED light starts to blink and you can find the backup in the .kobo folder.

How to install it: Check the instructions here.
How to repair your db: if you want to restore the backup, you have to follow these steps very carefully:

1. attach your device to a PC
2. go to .kobo directory
3. rename KoboReader.sqlite as you want
4. extract the archive KoboReader.sqlite.bz2 using an archive manager, like 7-Zip

-------------------------------------------------------------------------------

Currently the script has some limitations:

1. I need a way to detected wifi connection
2. I want to start the backup immediately if Kobo is sleeping, so I need a way to check sleeping status

I hope it will help us to save some time. Thanks to tshering for sqlite, LED control and USB flashing detection, and yoq for runonce trick.

[tshering]

Originally Posted by Lucas Malor

Since I got the database corrupted very much times, I decided to do this little auto-backup tool. It simply backups the db every 20 minutes, if it's not corrupted.

Currently the script has some limitations:[list=1][*] backup file is not in the flash partition, but in /var/bak/. This is the reason. I need a way to detected when the device is connected via USB

I think the following lines from /usr/local/Kobo/udev/usb might be helpful.

MODULE_LOADED=`lsmod | grep -c g_file_storage`

if [ $MODULE_LOADED -eq 0 ]; then
	exit
fi

[tshering]

Originally Posted by Lucas Malor

no message is displayed if db is corrupted. I know that I can display images using pickel showpic or pickel wait-for-hit (no pickel legendary?) but it does not work: I tried this while SSH connected, without any success:

PHP Code:

killall -STOP nickel
zcat /etc/images/error_kernel.raw.gz | /usr/local/Kobo/pickel wait-for-hit 0 0 100 100 0 100 100 100 


Did you try another picture? I think the format of error_kernel.raw.gz is not meant for the Glow?

[Lucas Malor]

Thank you tshering! I updated the tool, adding also a more reliable method to install it and removing the startup delay with an ad-hoc hack.
Now if the database gets corrupted it will be copied on the flash partition. The only problem with this method is that it's not reliable in all cases. For example when you attach the device using telnet or SSH, it won't work. I modified it a little for SSH case, but I would try to port lsusb to Kobo.

About pickel, I tried with

PHP Code:

zcat /etc/images/kraken-factory-reset.raw.gz | /usr/local/Kobo/pickel showpic 


but nothing. Furthermore when I send the CONTINUE signal to the nickel process, it becomes unresponsive, screen gets weird and I have to reboot.

Do you know if there's a way to control the LED light instead? I'll prefer it a lot much, but pickel blinkon seems to not work. Scripts in /etc/init.d seem to not control the LED.

[tshering]

I didn't try myself to control the LED apart from switching it off, but these two links should give enough information:
http://uni.2ch.net/test/read.cgi/ebooks/1355887854/544
http://www.tablix.org/~avian/blog/ar...linken_kindle/

[Lucas Malor]

Thank you very much! Now Kobo blinks if the db is corrupted. Furthermore I fixed some bugs and the backup is compressed now, so we save some space.

In the meanwhile I compiled a program that sends ioctl signals to devices. I do not need it anymore, but if someone is interested I attach the binary.

[TechniSol]

I don't want to rain on anyone's parade, hardly qualify as a fanboi of any denomination, but have to wonder just what you're doing that necessitates backing up the sqlite database every twenty minutes, and wonder that you might consider disclaiming the necessity for anyone using the reader as intended. I think we get enough people trolling without possibly seeming to indicate that the system is so fragile the db needs to be backed up every time we turn around... I sometimes wonder that Kobo provides enough ammunition just by updating so often because some people are afraid of breaking something that is working...

Anyhow, cool hack!

[Lucas Malor]

Originally Posted by TechniSol

what you're doing that necessitates backing up the sqlite database every twenty minutes

Backup is not made if db is not changed. 20 minutes are the minimum time between two backups, not the maximum.

Originally Posted by TechniSol

you might consider disclaiming the necessity for anyone using the reader as intended [...] I think we get enough people trolling without possibly seeming to indicate that the system is so fragile the db needs to be backed up every time we turn around

Well, I'm not a Kobo employee... and anyway, since I bought it, I think I can jump on it if I want to
I don't know if my db corruptions are my fault or it's the software that is fragile, because source code of nickel is closed. Saying that bugs are my fault is not 100% sure as saying it's Kobo's fault. Any guess is just... guess. I simply had a problem and I'm trying to do a workaround.

[tshering]

Originally Posted by TechniSol

..., but have to wonder just what you're doing that necessitates backing up the sqlite database every twenty minutes

This is a thing I wonder too. During the last two or three years I had only once a corrupted db, and that was my own doing. It happened however several times that after a sync with the kobo server the database was in a state I did not like (deleted book entries that lead finally to the deletion of files for instance). Such things however are unlikely to be detected by an integrity check. Another point is that I like to have snapshots of my db in order to preserve annotations of books that I might eventually delete from the device.
So for my needs it is better to do the backup at strategical points, before a synchronization, before deleting books from my device, before testing hacks etc.
Nevertheless, I like the project.

[Lucas Malor]

Originally Posted by tshering

for my needs it is better to do the backup at strategical points, before a synchronization, before deleting books from my device, before testing hacks etc.

Testing: I can start the backup immediately when usb is connected.
Deletion: I'll create a launcher for Kobo Start Monitor.
Sync: I have to find a way to detect internet connections...

[tshering]

Originally Posted by Lucas Malor

Testing: I can start the backup immediately when usb is connected.
Deletion: I'll create a launcher for Kobo Start Monitor.
Sync: I have to find a way to detect internet connections...

This is very kind. But please don't misunderstand me. I did not intend to ask for this feature. Actually, I don't need auto backup. I like to decide at what point I pack-up the database. I do it already with the Kobo Start Menu (scripts > back up > back up db.sh)

[TechniSol]

No offense intended, Lucas. I'm just trying to get you to look at the bigger picture. As far as stating that it backs up every twenty minutes, you did that in your first post:

"It simply backups the db every 20 minutes in a hidden place."

I suppose what you meant to say was, "It checks to see if the db has changed every twenty minutes and backs it up as long as it hasn't been corrupted in the mean time."

Of course, I'm just supposing... I'm also supposing it suspends during sleep mode or it would be a fair battery drain over time. Complete sentences composed with the same logic one devotes to a problem would be helpful in determining intent.

I don't work for Kobo, but I would like to give them a fair shake. I'm hard enough on them when they screw up, so the least I can do is try to be fair when they haven't.
I've had a great deal of enjoyment from the Touch and Glo I own, so feel they haven't screwed up too badly -mostly cosmetics and not listening when users want choices. I've also had a corrupted database maybe only once or twice in a couple of years and those occurred during copying of files or perhaps due to an improper dismount from windows. All that was required was to log out/in and let it rebuild the database and then replace any titles affected. Not a big deal. Annoying at worst as a non regularly scheduled inconvenience...

[Lucas Malor]

Updated, now backup starts also when an USB cable is attached to Kobo.

If someone is interested, I created a stupid little program that check the usb and a stupid little script that controls the led lights very easily. See the attachment.

Originally Posted by TechniSol

I'm also supposing it suspends during sleep mode or it would be a fair battery drain over time.

You supposed wrong. Currently I do not know how to check sleeping status. I think I must detect nickel process status, or the ability to catch touch signals. Anyway "fair battery drain" is an exageration. Do you ever checked the battery drain panel of any Android device? Usually more than 50% of power consumption is due to the use of the screen.

[TechniSol]

This is not an android device, e-ink displays use much less power than active TFT or the like, which is why we recharge after around 46.5 hours of actual reading/operation over a 24 day+ period with a Glo rather than 6-10 or so hours with the average Android tablet... at least according to the actual usage test I logged/performed with my Glo last year.(that was with lighting around 20% brightness as I recall)

All I was saying was I assumed you'd have a way to keep it from coming out of sleep every 20 minutes or that your code/timer wouldn't be executing because it was "powered down". If it is running every twenty minutes and impacting sleep mode then it's drawing current that need not be drawn because nothing is changing while the device is being slept. E-ink is all about low power. Granted it may not be a big impact if it's just checking for db change when there can be none, but every milliamp counts... Have you noticed how the average current draw drops off after a few seconds of no change while the device is running with only the normal code executing? This is why people had such trouble when using sleep covers that had a pocket that tripped the IR hardware into a powered mode by blocking the IR and preventing a proper sleep mode. Typical battery life in that situation was under 72 hours in what people thought was sleep.

I think you're doing great, keep going.

[Lucas Malor]

Ok, I updated the tool. Now it backups the database immediately if an internet connection is established.

@TechniSol: you can check yourself the cpu usage Just ssh your Kobo and do:

PHP Code:

top | grep bak 


On my device, when it doesn't backup the db, cpu usage is 0.4% at max. he tool reaches this cpu usage only every 3 seconds.

I can't find a way to detect sleeping status, but I suppose it doesn't really matter: db can't change while Kobo is sleeping, so the auto-backup tool remains at that level of cpu usage.