Wish for Device Driver mount point

[PeterT]

Something I've often wondered about (and wished for) is the ability to rather than rely on calibre detecting the USB mount of a device, for it to "optionally" / "experimentally" detect a given mounting point / driver letter as being a device.

This wish is driven by the fact that my main computer these days is a chromebook which while it DOES support access to USB storage does NOT pass the USB device through to linux environment; instead USB storage is presented as /mnt/chromeos/removable/.... ; in my case as /mnt/chromeos/removable/KOBOeReader/

Of course I can do a "save to disk", or a "copy" at the O/S level, access calibre via the Calibre Content Server or transfer content via DropBox / gDrive, but it would be oh so nice to have the abiity to specify a location for the Kobo as a configurable item for the device driver.

[chaley]

If I understand you correctly, you want to do what "Connect to folder" already does but for other devices?

One problem that wouldn't be easy to solve: the USB information helps calibre know which variant of the device is being used. Whatever mechanism used to find the device would also need to somehow know the variant, which implies some kind of externally-maintained (by you) mapping from USB info -> mount point or vice versa

Another approach would be to allow (somehow) a mapping from an abritrary folder to a device variant. This avoids detecting mounted devices, instead depending on the existence of a path to the device root. The mapping would need to be unique or one might have both the kindle and kobo drivers fighting over the same folder.

[PeterT]

Yes. The idea would be (for instance) to allow manual selection of a device model and mount point so that, for instance, the Kobo driver could correctly access the on device database, and that the Kobo Utilities could also take backups etc.

Yes while I could connect using "connect to folder" this does not allow the Kobo specific functions to work.

I know from looking through both the driver and utility plugin that davidfor did have some hard coded paths he could use for testing without the device actually connected.

Sent from my Pixel 7a using Tapatalk

[Quoth]

Originally Posted by PeterT

Something I've often wondered about (and wished for) is the ability to rather than rely on calibre detecting the USB mount of a device, for it to "optionally" / "experimentally" detect a given mounting point / driver letter as being a device.

This wish is driven by the fact that my main computer these days is a chromebook which while it DOES support access to USB storage does NOT pass the USB device through to linux environment; instead USB storage is presented as /mnt/chromeos/removable/.... ; in my case as /mnt/chromeos/removable/KOBOeReader/

I saner wish (but no-one at Google listens) is that if you don't enable the device for Android in External Storage, and do add it as a USB Device (and that option does appear) to Linux Development system, it will work as it should. Sometimes (once on MTP and once with USB-MS) that did work with Calibre on Crostini.

USB support in Crostini seems to be in flux. It seems very hit & miss as to which "supported devices" appearing on the Linux USB Manage screen actually work in Linux (Crostini is currently a minimal Debian 12 with Wayland, but Chrome Browser appears to render the windows).

Lots of things that work now on Crostini didn't use to work. I can mount SAMBA shares in Crostini instead of via Google Files and Share with Linux.

An ext4 microSD card works.

My main PC is Linux Mint + Mate on a Dell 7050 on Solar UPS with 23" 4K UHD screen, 512K SSD & 4T HDD. I have a 17" HD Lenovo laptop with 256 SSD & 2T HDD for "going out", but the Samsung Galaxy Go Chromebox has about twice the battery run time and I now maybe have everything I want except the Compose Key. There is a soon to be expunged ComposeKey extension via ChromeBrowser that works on Android and ChromeOS, inc the "penguin" terminal, but not on any Linux App. AFAIK iBus or any Linux IME doesn't work and the input is magically from ChromeOS. Setting Regional in Linux to be UK/Ireland rather than USA was a bit tricky.

I can more easily connect to Kindle on actual Android than Crostini!

It's a moving target. If someone did what you want today it would break tomorrow!

Also neither Android, ChromeOS, Crostini or regular Linux has drive letters.

[kovidgoyal]

One could use a user maintained mapping of filesystem path to usb vendor/product/bcd ids to impleent this, with a fake device being detected and the appropriate driver being called. But its a fair bit of work for a rather niche use case so not sure I want to spend the time.

There will be some subtleties like how to detect disconnect. And of course it will only work for USBMS based devices, not MTP based ones.

[Quoth]

Originally Posted by kovidgoyal

One could use a user maintained mapping of filesystem path to usb vendor/product/bcd ids to impleent this, with a fake device being detected and the appropriate driver being called. But its a fair bit of work for a rather niche use case so not sure I want to spend the time.

There will be some subtleties like how to detect disconnect. And of course it will only work for USBMS based devices, not MTP based ones.

I rather suspect that given the progress on Crostini since (amazingly for google) that it exited Beta, that sooner or later both USBMS & MTP will work normally in the Linux container. They sometimes do now if the device isn't ebabbled for Android app in External Storage in regular settings.

[PeterT]

I seriously doubt Google will change how USB devices are shared. If you check any of the long standing bug reports on this the standard answer is it's done that way for security.

Sent from my Pixel 7a using Tapatalk

[Quoth]

Well, no, Crostini Linux is a development environment, not for the masses like Android or Chrome OS itself.

[Quoth]

Originally Posted by PeterT

I seriously doubt Google will change how USB devices are shared. If you check any of the long standing bug reports on this the standard answer is it's done that way for security.

Sent from my Pixel 7a using Tapatalk

Researching this it's not actually about security on Crostini, as that's deemed to be a development environment. Because the Crostini Debian 12 is in a container, anyway, the normal way of mounting a USB Mass Storage device doesn't apply. Currently from my research and reading Google edicts, it seems only Android USB MTP can be passed through. Tools like Timeshift can see a USB HDD enabled under Linux USB Devices, but can't mount it, unsurprisingly.

Guest Persistence under Linux USB settings needs enabled to remember an MTP Android, and if you add any USB device to Linux you can't easily get it back on ChromeOS & Android by turning off. You need to disable the USB device, then disable and re-enable USB persistence so that the next time the device is plugged in to USB that it's available to External Storage in Settings and Google File Manager.

I've no Kindle MTP to test (and don't want one). Right now for an ereader using USB Mass Storage you should not enable it in Linux but let Google File manager share it. The ext4 format as well as FAT32, exFAT and NTFS work. Then Caja (or whatever) will move rather than copy unless you hold Control when drag & Drop. Linux utilities to mount network shares work and this is preferable to using Google File Manager for Network shares.

So I can't see how Kovid can change the behaviour. The USB Mass Storage ereaders have to be shared from ChromeOS and treated as directories (folders) in Calibre.

There is a possibility of other ways if full developer mode is enabled with expanded crosh features. Doing that erases everything, but you made backups?

There is a way to make USB storage and other devices available in the container. It sort of works in Oracle Vbox, though curiously the Sony program on XP in a VM couldn't access the Sony PRS-350 ereader. Plenty of other stuff on USB works.

Edit:
I installed Chrome Browser ComposeKey extension and it works on ChromeOS, Android and the Crostini Penguin terminal.
There are flags for "virtual keyboard" and QT support for crostini in Chrome Browser, and then the Compose Key worked in Pluma, but not in LO Writer or KATE (lack of IME / ibus likely).

[kovidgoyal]

This is now implemented in master. The connect to folder function allows you to optionally pick a specific device model (from a list of USBMS devices) to connect as.

Note its only very lightly tested so no idea how well it actually works.

[Quoth]

Originally Posted by kovidgoyal

This is now implemented in master. The connect to folder function allows you to optionally pick a specific device model to connect as.

Note its only very lightly tested so no idea how well it actually works.

I'll test it later. Thanks, though I usually use a "real" Linux rather than Crostini. The MTP direct to Crostini Linux running Calibre actually works better than ordinary native Linux. Only tested with Android phone and tablet. So currently this is only needed with USB Mass Storage. Direct USB of a DVB-T stick works for SDR in Linux and Direct USB of Sony NetMD minidisc also works. I've not tried the FTDI USB serial adaptor yet, or my USB Centronics parallell port.

[PeterT]

Originally Posted by kovidgoyal

This is now implemented in master. The connect to folder function allows you to optionally pick a specific device model (from a list of USBMS devices) to connect as.

Note its only very lightly tested so no idea how well it actually works.

I look forward to trying it Thanks for implementing this.

[Quoth]

Originally Posted by Quoth

I'll test it later. Thanks, though I usually use a "real" Linux rather than Crostini. The MTP direct to Crostini Linux running Calibre actually works better than ordinary native Linux. Only tested with Android phone and tablet. So currently this is only needed with USB Mass Storage. Direct USB of a DVB-T stick works for SDR in Linux and Direct USB of Sony NetMD minidisc also works. I've not tried the FTDI USB serial adaptor yet, or my USB Centronics parallell port.

FTDI USB Serial also works direct in Crostini (with PuTTY, though you need to select an actually installed font first!).

[PeterT]

Originally Posted by kovidgoyal

This is now implemented in master. The connect to folder function allows you to optionally pick a specific device model (from a list of USBMS devices) to connect as.

Note its only very lightly tested so no idea how well it actually works.

Well.. Mixed results

I plugged the Kobo Clara Colour in via USB, selected the path and device type and....

calibre, version 7.26.101
ERROR: Error: Error communicating with device

IOError: disk I/O error

Traceback (most recent call last):
  File "/home/peterjt/work/calibre-master/src/calibre/gui2/device.py", line 111, in run
    self.result = self.func(*self.args, **self.kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peterjt/work/calibre-master/src/calibre/gui2/device.py", line 582, in _books
    mainlist = self.device.books(oncard=None, end_session=False)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peterjt/work/calibre-master/src/calibre/devices/kobo/driver.py", line 2023, in books
    self.dbversion = self.get_database_version(connection)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/peterjt/work/calibre-master/src/calibre/devices/kobo/driver.py", line 200, in get_database_version
    cursor.execute('SELECT version FROM dbversion')
  File "src/cursor.c", line 1088, in APSWCursor_execute.sqlite3_prepare_v3
apsw.IOError: IOError: disk I/O error

Just in case it was a corrupted database I did what I normally do and cloned the Kobo to the local hard drive

rclone -v sync /mnt/chromeos/removable/KOBOeReader/ libra/

and tried a pragma integrity check on the database; no issues.

I now tried connecting to the local folder and... TADA. It worked!

So obviously there is something wonky in how the chromebook mounts USB storage devices and shares them. I know it uses 9p to share the mount points

mount | grep 9p
9p on /mnt/chromeos type 9p (rw,nosuid,nodev,relatime,access=any,msize=4120,trans=fd,rfd=15,wfd=15)

so there is probably something there

[PeterT]

Additionally it looks as if updates will be needed to some of the Kobo Plugins. I tried using Kobo Utilities to compress the (mirrored to local) copy of the database

calibre, version 7.26.101
ERROR: Unhandled exception: <b>CantOpenError</b>:CantOpenError: unable to open database file

calibre 7.26.101*  embedded-python: True
Linux-6.6.65-06377-gaae6fc9ba7df-x86_64-with-glibc2.36 Linux ('64bit', 'ELF')
('Linux', '6.6.65-06377-gaae6fc9ba7df', '#1 SMP PREEMPT_DYNAMIC Wed, 8 Jan 2025 13:19:21 -0800')
Python 3.11.5
Interface language: None
EXE path: /opt/calibre/bin/calibre
Successfully initialized third party plugins: DeACSM (0, 0, 16) && LCPL Input (0, 0, 5) && DeDRM (10, 0, 9) && Annotations (1, 17, 13) && Backup Configuration Folder (1, 1, 2) && EpubMerge (3, 2, 0) && Find Duplicates (1, 10, 9) && Goodreads (1, 8, 2) && Kobo Metadata (1, 10, 0) && Kobo Utilities (2, 18, 4) && Manage Series (1, 5, 2) && OverDrive Libby (0, 2, 3) && Quality Check (1, 13, 16) && Reading List (1, 15, 4) && Standard Ebooks (1, 0, 0)
Traceback (most recent call last):
  File "calibre_plugins.koboutilities.action", line 2136, in check_device_database
    check_result = self._check_device_database()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.koboutilities.action", line 4582, in _check_device_database
    return check_device_database(self.device_database_path())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.koboutilities.common_utils", line 301, in check_device_database
    with device_database_connection(database_path) as connection:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.koboutilities.common_utils", line 293, in device_database_connection
    db_connection = apsw.Connection(database_path)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
apsw.CantOpenError: CantOpenError: unable to open database file

Likewise the backup on connect resulted in

calibre, version 7.26.101 (linux, embedded-python: True)
Failed to backup device database: Failed: Backing up Kobo device database

[Errno 2] No such file or directory: '/home/peterjt/libra.kobo/KoboReader.sqlite'

Traceback (most recent call last):
  File "/home/peterjt/work/calibre-master/src/calibre/gui2/device.py", line 111, in run
    self.result = self.func(*self.args, **self.kwargs)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "calibre_plugins.koboutilities.jobs", line 129, in do_device_database_backup
    shutil.copyfile(database_file, backup_file_path)
  File "shutil.py", line 256, in copyfile
FileNotFoundError: [Errno 2] No such file or directory: '/home/peterjt/libra.kobo/KoboReader.sqlite'

HMMM.. The path being used is/home/peterjt/libra.kobo/KoboReader.sqlite whereas the ACTUAL path is '/home/peterjt/libra/.kobo/KoboReader.sqlite; looks as if a character is being trimmed from the path.