Based on my assumption, I believe I have identified both the problem and a valid workaround.
I monitored the behavior of nickel during a USB connection in relation to the sqlite database. The problem is that nickel does not reliquish its control over the sqlite database during a usb connection, keeping the file open. Note that after disconnecting the USB nickel actually closes and reopens the database.
EDIT:
Nickel only closes the DB if it "processes contents". Otherwise a reboot is necessary to get it reopened. sry about that
END EDIT
The problem with this behavior is apparent: SQLite is a single process database implementing filesystem level locks. Obviously on the used FAT filesystem, that does not work properly. The KT driver in calibre should never write to the database at all, as it is opened by nickel while the usb is connected and any UPDATE/INSERT commands risk corrupting the database.
Linux uses filedescriptors based on FS handles. The thing exploited here is that the handle stays the same, even if the filename changes, since the filename is only an attribute of the FS handle. As nickel opens the database before the USB connection is established, it does not really care what the file is called after that - it is safe to rename it (though not to move it to another directory, just renaming). So if the KT driver actually writes to a different SQLite database that is not open (but named KoboReader.sqlite), the problem is solved.
Heres the workaround:
- Connect the KT to the computer. Do not start calibre!
- Rename the KoboReader.sqlite database to something like KoboReader.sqlite.open. The filedescriptor held by nickel will be redirected to KoboReader.sqlite.open, so we don't actually care what nickel writes back.
- Copy KoboReader.sqlite.open to KoboReader.sqlite. This is the file instance manipulated by calibre.
- Start calibre and have the metadata uploaded.
- Disconnect and restart the KT (forces nickel to open the KoboReader.sqlite instead of the KoboReader.sqlite.open).
- Optionally close calibre, reconnect the KT to the computer and remove KoboReader.sqlite.open.
That way the database updated by the KT driver is closed and not being written to by nickel upon disconnecting the device.
Please note: I have only tested this procedure for updating metadata. But I suppose it should also work with sending ebooks to the device, as long the procedure is repeated before every calibre interaction with the database.