Wish for Device Driver mount point - Page 4

kovidgoyal · 03-10-2025, 10:09 PM

device driver functions are called in a single non-GUI thread. Not to mention that there is only one temporary location per driver instance.

kovidgoyal · 03-10-2025, 10:38 PM

Added some code to handle the WAL journal:
https://github.com/kovidgoyal/calibr...32fd54650ffb54

It's extremely unfortunate Kobo turned on WAL for the databases. It's not like they have high concurrency and hence need it.

Majutsushi · 03-11-2025, 01:35 AM

Quote:

Originally Posted by kovidgoyal

device driver functions are called in a single non-GUI thread. Not to mention that there is only one temporary location per driver instance.

What about plugins? I'm particularly thinking of the Kobo Utilities plugin, which has some feature that allow users to modify the device database, and which currently uses its own database connections. Is there a way to make sure that those features don't run at the same time as other device driver functions? I'm not very familiar with the Calibre internals so I'm not sure how likely this scenario is.

kovidgoyal · 03-11-2025, 02:03 AM

There you go: https://github.com/kovidgoyal/calibr...25dce149ad5bfc

Majutsushi · 03-11-2025, 02:10 AM

Quote:

Originally Posted by kovidgoyal

There you go: https://github.com/kovidgoyal/calibr...25dce149ad5bfc

Thanks!

Majutsushi · 03-16-2025, 12:18 AM

Sorry to bring this up again, but after looking into this more I think there is still an issue. As far as I can tell background jobs are executed as separate processes, so if a plugin accesses the database in such a job it won't have access to the lock of the main process, and so it can't coordinate with accesses from the main process.

Is there anything that can be done about this? The only thing I can think of are either file-based locks or changing the plugin to not use background jobs if the database has been copied and instead do everything in the foreground.

kovidgoyal · 03-16-2025, 07:13 AM

device jobs in calibre are in the main process. In a plugin it's up to you to decide if you want thread or process based background jobs. The existing lock is single process and I dont think I want to make it an IPC lock as the overhead for that is too large.

So either use thread based background jobs in the plugin or simply get a write lock on the temp db path in your plugin. You can do so by opening a sqline connection to it and starting a trnsaction. sqlite IIRC is single writer so that should serve as a fine IPC lock.

03-10-2025, 10:09 PM	#46
kovidgoyal creator of calibre Posts: 45,359 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	device driver functions are called in a single non-GUI thread. Not to mention that there is only one temporary location per driver instance. Last edited by kovidgoyal; 03-10-2025 at 10:45 PM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
boox-wifi-mount-sync - sync or mount your Boox device over WiFi	mag_dex	Onyx Boox	3	07-30-2022 01:42 PM
Aura H2O 2 Mount Kobo device without access to screen	7596ff	Kobo Reader	9	02-21-2020 11:51 AM
Which block device are USB drives on the 350? (To mount)	Ami_K	Sony Reader Dev Corner	0	01-08-2019 02:16 AM
unable to mount any device in 1.3	boscaiolo	Devices	4	09-16-2013 05:02 AM
calibre assumes incorrect location of reader mount point?	akos.maroy	Library Management	8	12-19-2011 06:34 AM

03-10-2025, 10:38 PM	#47
kovidgoyal creator of calibre Posts: 45,359 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Added some code to handle the WAL journal: https://github.com/kovidgoyal/calibr...32fd54650ffb54 It's extremely unfortunate Kobo turned on WAL for the databases. It's not like they have high concurrency and hence need it.

03-11-2025, 02:03 AM	#49
kovidgoyal creator of calibre Posts: 45,359 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	There you go: https://github.com/kovidgoyal/calibr...25dce149ad5bfc

03-16-2025, 12:18 AM	#51
Majutsushi Zealot Posts: 100 Karma: 8592 Join Date: Aug 2023 Location: New Zealand Device: Kobo Clara HD, Kobo Libra Colour	Sorry to bring this up again, but after looking into this more I think there is still an issue. As far as I can tell background jobs are executed as separate processes, so if a plugin accesses the database in such a job it won't have access to the lock of the main process, and so it can't coordinate with accesses from the main process. Is there anything that can be done about this? The only thing I can think of are either file-based locks or changing the plugin to not use background jobs if the database has been copied and instead do everything in the foreground.

03-16-2025, 07:13 AM	#52
kovidgoyal creator of calibre Posts: 45,359 Karma: 27182818 Join Date: Oct 2006 Location: Mumbai, India Device: Various	device jobs in calibre are in the main process. In a plugin it's up to you to decide if you want thread or process based background jobs. The existing lock is single process and I dont think I want to make it an IPC lock as the overhead for that is too large. So either use thread based background jobs in the plugin or simply get a write lock on the temp db path in your plugin. You can do so by opening a sqline connection to it and starting a trnsaction. sqlite IIRC is single writer so that should serve as a fine IPC lock.

Advert

Advert