Wish for Device Driver mount point

[kovidgoyal]

This should allow the driver to work with 9p filesystems as well.

https://github.com/kovidgoyal/calibr...e3aef3a798650a

[Quoth]

Originally Posted by kovidgoyal

This should allow the driver to work with 9p filesystems as well.

https://github.com/kovidgoyal/calibr...816e2b1208b2b8

Thanks, you are a star.
Edited link

[PeterT]

Originally Posted by kovidgoyal

This should allow the driver to work with 9p filesystems as well.

https://github.com/kovidgoyal/calibr...e3aef3a798650a

Didn't seem to work. See the log

Spoiler:

ApplicationPaletteChange event ignored
Using calibre Qt style: True
Failed to load resource: 'commit.txt' from the plugin zip file: /home/peterjt/.config/calibre/plugins/OverDrive Libby.zip
Traceback (most recent call last):
  File "/home/peterjt/work/calibre-master/src/calibre/customize/zipplugin.py", line 46, in get_resources
    ans[name] = zf.read(name)
                ^^^^^^^^^^^^^
  File "zipfile.py", line 1510, in read
  File "zipfile.py", line 1547, in open
  File "zipfile.py", line 1476, in getinfo
KeyError: "There is no item named 'commit.txt' in the archive"
[overdrive_libby/0.2.3] Loaded 1 items from file cache /home/peterjt/.config/calibre/plugins/overdrive_libby.libraries.json
[overdrive_libby/0.2.3] Loaded 0 items from file cache /home/peterjt/.config/calibre/plugins/overdrive_libby.media.json
Failed to create system tray icon, your desktop environment probably does not support the StatusNotifier spec https://www.freedesktop.org/wiki/Specifications/StatusNotifierItem/
EpubMerge: DEBUG: 2025-03-08 09:47:16,518: calibre_plugins.epubmerge.epubmerge_plugin(156): macmenuhack file_path:/home/peterjt/.config/calibre/plugins/fanficfare_macmenuhack.txt
QRhiGles2: Failed to create temporary context
QRhiGles2: Failed to create context
This plugin does not support createPlatformVulkanInstance
QVulkanInstance: Failed to initialize Vulkan
Unable to detect GPU vendor.
WebEngineContext is used before QtWebEngineQuick::initialize() or OpenGL context creation failed.
calibre Debug log
calibre 7.26.102*  embedded-python: True
Linux-6.6.65-06377-gaae6fc9ba7df-x86_64-with-glibc2.36 Linux ('64bit', 'ELF')
('Linux', '6.6.65-06377-gaae6fc9ba7df', '#1 SMP PREEMPT_DYNAMIC Wed, 8 Jan 2025 13:19:21 -0800')
Python 3.11.5
Interface language: None
EXE path: /opt/calibre/bin/calibre-debug
Successfully initialized third party plugins: DeACSM (0, 0, 16) && LCPL Input (0, 0, 5) && DeDRM (10, 0, 9) && Annotations (1, 17, 13) && Backup Configuration Folder (1, 1, 2) && EpubMerge (3, 2, 0) && Find Duplicates (1, 10, 9) && Goodreads (1, 8, 2) && Kobo Metadata (1, 10, 0) && Kobo Utilities (2, 18, 4) && Manage Series (1, 5, 2) && OverDrive Libby (0, 2, 3) && Quality Check (1, 13, 16) && Reading List (1, 15, 4) && Standard Ebooks (1, 0, 0)
calibre 7.26.102*  embedded-python: True
Linux-6.6.65-06377-gaae6fc9ba7df-x86_64-with-glibc2.36 Linux ('64bit', 'ELF')
('Linux', '6.6.65-06377-gaae6fc9ba7df', '#1 SMP PREEMPT_DYNAMIC Wed, 8 Jan 2025 13:19:21 -0800')
Python 3.11.5
Interface language: None
EXE path: /opt/calibre/bin/calibre-debug
Successfully initialized third party plugins: DeACSM (0, 0, 16) && LCPL Input (0, 0, 5) && DeDRM (10, 0, 9) && Annotations (1, 17, 13) && Backup Configuration Folder (1, 1, 2) && EpubMerge (3, 2, 0) && Find Duplicates (1, 10, 9) && Goodreads (1, 8, 2) && Kobo Metadata (1, 10, 0) && Kobo Utilities (2, 18, 4) && Manage Series (1, 5, 2) && OverDrive Libby (0, 2, 3) && Quality Check (1, 13, 16) && Reading List (1, 15, 4) && Standard Ebooks (1, 0, 0)
QPA platform: xcb
devicePixelRatio: 1.25
logicalDpi: 96.0 x 96.0
physicalDpi: 96.09458128078816 x 96.25263157894736
[0.00] Starting up...
[0.00] Showing splash screen...
[1.05] splash screen shown
[1.06] Initializing db...
[1.31] db initialized
[1.31] Constructing main UI...
DEBUG:    0.0 Annotations plugin: Error loading libiMobileDevice. This hasn't worked for a while, and is blacklisted in calibre v3.
DEBUG:    0.0 Annotations plugin: Error is:  No module named 'calibre.devices.idevice'
DEBUG:    0.0 Annotations::reader_app_support.py - loading translations
DEBUG:    0.0 Annotations::common_utils.py - loading translations
DEBUG:    0.0 Annotations::appearance.py - loading translations
DEBUG:    0.0 Annotations::config.py - loading translations
DEBUG:    0.0 Annotations::annotated_books.py - loading translations
DEBUG:    0.0 Annotations::action.py - loading translations
DEBUG:    0.0 Kobo Utilities Configuration - CreateNewCustomColumn is supported
DEBUG:    0.6  ~~~~~~~~~~~~~~~~~~~~~~ Annotations plugin 1.17.13 (linux) ~~~~~~~~~~~~~~~~~~~~~~
DEBUG:    0.6 AnnotationsAction:genesis(ERROR) Error loading library libiMobileDevice: name 'libiMobileDevice' is not defined
DEBUG:    0.6 AnnotationsAction:init_options(disable_caching: False) 
DEBUG:    0.6 AnnotationsDB:connect(db_version: 1) 
DEBUG:    0.6 AnnotationsAction:init_prefs(Updating prefs to 1.17.13) 
DEBUG:    0.6 AnnotationsAction:load_dynamic_reader_classes() 
DEBUG:    0.6   loading built-in class 'BooxReaderApp'
DEBUG:    0.6   loading built-in class 'GoodReader'
DEBUG:    0.6   loading built-in class 'Kindle'
DEBUG:    0.6   loading built-in class 'KoboTouch'
DEBUG:    0.6   loading built-in class 'ParseKindleMyClippingsTxt'
DEBUG:    0.6   loading built-in class 'ParseTolinoNotesTxt'
DEBUG:    0.6   loading built-in class 'PocketBook'
DEBUG:    0.6   loading built-in class 'Sony'
DEBUG:    0.6   loading built-in class 'Tolino'
DEBUG:    0.6 AnnotationsAction:inflate_dialog_resources() 
[2.55] GUI main window shown
DEBUG:    1.0 No Kobo Touch, Glo or Mini appears to be connected
DEBUG:    1.0 rebuild_menus - self.supports_ratings=None, self.supports_tiles=None
DEBUG:    1.0 KoboUtilities:set_toolbar_button_tooltip - start: text='None'
DEBUG:    1.0 device_driver_name - could not load extended driver. Exception= No module named 'calibre_plugins.kobotouch_extended'
DEBUG:    1.0 KoboUtilities:set_toolbar_button_tooltip - setting to text='Utilities to use with Kobo ereaders

Driver: KoboTouch'
DEBUG:    1.1 device_driver_name - could not load extended driver. Exception= No module named 'calibre_plugins.kobotouch_extended'
[2.63] main UI initialized...
[2.63] Hiding splash screen
[2.63] splash screen hidden
[2.63] Started up in 2.63 seconds with 2830 books
[2.63] Starting the smartdevice driver
SMART_DEV (   0.00:  0.001) _attach_to_port try ip address "" on port 9090
SMART_DEV (   0.88:  0.823) _startup_on_demand listening on port 9090
SMART_DEV (   0.89:  0.000) _attach_to_port try ip address "" on port 54982
SMART_DEV (   0.90:  0.000) _startup_on_demand broadcast socket listening on port 54982
[3.54] Finished starting smartdevice
SMART_DEV (   1.90:  1.001) run All IP addresses {'lo': [{'addr': '127.0.0.1', 'netmask': '255.0.0.0', 'peer': '127.0.0.1'}], 'docker0': [{'addr': '172.17.0.1', 'netmask': '255.255.0.0', 'broadcast': '172.17.255.255'}], 'eth0': [{'addr': '100.115.92.205', 'netmask': '255.255.255.240', 'broadcast': '100.115.92.207'}]}
Unable to open device Driver:KoboTouch, Options - format_map: ['kepub', 'epub', 'pdf', 'txt', 'cbz', 'cbr'], use_subdirs: True, read_metadata: True, use_author_sort: False, save_template: {author_sort}/{title} - {authors}, extra_customization: [], manage_collections: True, use_collections_columns: True, collections_columns: , use_collections_template: False, collections_template: , create_collections: False, delete_empty_collections: False, ignore_collections_names: , upload_covers: False, dithered_covers: False, keep_cover_aspect: False, upload_grayscale: False, letterbox_fs_covers: False, letterbox_fs_covers_color: #000000, png_covers: False, show_archived_books: False, show_previews: False, show_recommendations: False, update_series: True, force_series_id: False, update_core_metadata: False, update_purchased_kepubs: False, update_device_metadata: True, update_subtitle: False, subtitle_template: , update_bookstats: False, bookstats_wordcount_template: , bookstats_pagecount_template: , bookstats_timetoread_upper_template: , bookstats_timetoread_lower_template: , kepubify: True, template_for_kepubify: None, modify_css: False, per_device_css: {}, override_kobo_replace_existing: True, affect_hyphenation: False, disable_hyphenation: False, hyphenation_min_chars: 6, hyphenation_min_chars_before: 3, hyphenation_min_chars_after: 3, hyphenation_limit_lines: 2, support_newer_firmware: False, debugging_title: , driver_version: 2.6.0
Traceback (most recent call last):
  File "/home/peterjt/work/calibre-master/src/calibre/devices/kobo/db.py", line 52, in __init__
    connect()
  File "/home/peterjt/work/calibre-master/src/calibre/devices/kobo/db.py", line 43, in connect
    cursor.execute('SELECT version FROM dbversion')
  File "src/cursor.c", line 1088, in APSWCursor_execute.sqlite3_prepare_v3
apsw.IOError: IOError: disk I/O error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/peterjt/work/calibre-master/src/calibre/gui2/device.py", line 250, in do_connect
    dev.open(detected_device, self.current_library_uuid)
  File "/home/peterjt/work/calibre-master/src/calibre/devices/usbms/device.py", line 772, in open
    self.post_open_callback()
  File "/home/peterjt/work/calibre-master/src/calibre/devices/kobo/driver.py", line 1615, in post_open_callback
    self.db_manager = Database(self.device_database_path)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peterjt/work/calibre-master/src/calibre/devices/kobo/db.py", line 55, in __init__
    debug_print(f'Kobo: I/O error connecting to {self.device_database_path} copying it into temporary storage and connecting there')
                                                 ^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Database' object has no attribute 'device_database_path'

[95.30] Shutdown starting...
DEBUG:   93.9 AnnotationsAction:shutting_down() 
[95.54] [Backup Configuration Folder]: starting backup process
[97.37] [Backup Configuration Folder]: wrote config backup to /home/peterjt/work/backup/backup-calibre/2025-03-08 at 09-48-49.zip. Size: 37,442,988 bytes.
[97.37] [Backup Configuration Folder]: finished
[99.41] Shutdown complete, quitting...

Still works with the cloned copy on the local storage.

[kovidgoyal]

@PeterT: Stupid typo. https://github.com/kovidgoyal/calibr...816e2b1208b2b8

[PeterT]

Originally Posted by kovidgoyal

@PeterT: Stupid typo. https://github.com/kovidgoyal/calibr...816e2b1208b2b8

Still no go

Spoiler:

calibre, version 7.26.102
ERROR: Error: Error communicating with device

'apsw.Connection' object has no attribute 'cache_flush'

Traceback (most recent call last):
  File "/home/peterjt/work/calibre-master/src/calibre/gui2/device.py", line 111, in run
    self.result = self.func(*self.args, **self.kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peterjt/work/calibre-master/src/calibre/gui2/device.py", line 582, in _books
    mainlist = self.device.books(oncard=None, end_session=False)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peterjt/work/calibre-master/src/calibre/devices/kobo/driver.py", line 1993, in books
    with self.database_transaction(use_row_factory=True) as connection:
  File "/home/peterjt/work/calibre-master/src/calibre/devices/kobo/db.py", line 74, in __exit__
    copy_db(self.conn, self.path_on_device)
  File "/home/peterjt/work/calibre-master/src/calibre/devices/kobo/db.py", line 19, in copy_db
    conn.cache_flush()
    ^^^^^^^^^^^^^^^^
AttributeError: 'apsw.Connection' object has no attribute 'cache_flush'

[kovidgoyal]

Next try: https://github.com/kovidgoyal/calibr...a1b622749f1bb3

[PeterT]

That seemed to work. When I have more time I'll do more testing of plugins. I did try Kobo Utilities and it had problems with an apsw error on the Check Database option. I didn't have time to gather logs but will in a few hours.

Sent from my Pixel 7a using Tapatalk

[kovidgoyal]

I'm afraid getting third party plugins working is not really something I am going to spend time on, if they have issues you will need to report them to the developer of the plugin and hopefully he can adapt the approach I have used in his plugin.

[PeterT]

Originally Posted by kovidgoyal

I'm afraid getting third party plugins working is not really something I am going to spend time on, if they have issues you will need to report them to the developer of the plugin and hopefully he can adapt the approach I have used in his plugin.

That was my intention. Thanks for all your work on this idea though

Sent from my Pixel 7a using Tapatalk

[PeterT]

I bet by now you're getting quite tired of me! I've come across a really weird side effect.

I'm reading an Overdrive library loan fulfilled via the Kobo library. The book is listed on the device as being of type Overdrive. I connect to the device via calibre, then perform some operations (say send books to the Kobo) and disconnect.

When I unplug the Kobo the book now shows as being no on the device (yes; it shows up in the view of books on the device) however, it has an open arrow next it indicating that it's not physically on the device. I perform a sync with the Kobo cloud, the book downloads. Sometimes I'm at the correct location in the book; other times I've reverted to a previous reading location.

I'm going to have to do a lot more digging to track down what might be happening here.

[kovidgoyal]

Is this specific to using the Kobo via 9p or does it happen with a direct connection as well?

[PeterT]

I'll experiment tomorrow; I think I have access to a PC where I can try things out.

I did try some testing out today with just connecting calibre to the 9p folder and disconnecting with no usage of Kobo Utilities or automatic meta data management.

sqldiff showed no differences in the databases before connect and after disconnect.

The Kobo post disconnect also had a sqlite-shm and sqlite-wal file present.

[DNSB]

Originally Posted by PeterT

I'll experiment tomorrow; I think I have access to a PC where I can try things out.

I did try some testing out today with just connecting calibre to the 9p folder and disconnecting with no usage of Kobo Utilities or automatic meta data management.

sqldiff showed no differences in the databases before connect and after disconnect.

The Kobo post disconnect also had a sqlite-shm and sqlite-wal file present.

That would be more or less normal since when the database is in use, those files would be present.

[Quoth]

In case anyone is wondering?
9p is from Plan9

* Windows Subsystem for Linux (WSL): since Windows 10 version 1903, the subsystem implements 9P as a server and the host Windows operating system acts as a client.
* Crostini: a custom 9P server is used to provide access to files outside of a Linux VM.
* QEMU: the VirtFS device allows for filesystem sharing over 9P, which is accelerated with kernel drivers and shared memory

https://en.wikipedia.org/wiki/9P_(protocol)

So perhaps testing is also possible with WSL and Linux Calibre on WSL? Or maybe I'm writing rubbish.

Crostini is the Linux based on Debiam 12 in the "penguin" container on a ChromeBook, which works better than I thought it would.

[Majutsushi]

I may be missing something, but the way the Database class copies the database around doesn't seem at all threadsafe to me. What happens in the following situation:

• Thread A copies database to temporary location A and makes some changes
• Thread B copies database to temporary location B and makes some changes
• Thread A copies database back to the device
• Thread B copies database back to the device

The changes thread A made would be lost. Or is there something in the architecture of Calibre that prevents this situation from happening?