Kobo: Book not 'On Device' until after Kobo import?

[JimmXinu]

This isn't how I first noticed the issue, but it is the easiest to replicate. Here's what I'm seeing:

1. Connect Kobo via USB.
2. Send new epub (not kepub) book from Calibre to Kobo over USB.
3. Book shows 'Main' in 'On Device' column in Library view.
4. Quit Calibre -- without disconnecting Kobo.
5. Start Calibre -- wait for device to be recognized.
6. Book does NOT shows anything in 'On Device' column, nor in the Device view.

Following these steps, I can get everything back to expected normal:

1. Send book to Kobo again
2. Eject Kobo.
3. Wait for Kobo 'import books' to run and new book to appear in Kobo list.
4. Reconnect Kobo.
5. Book does show 'Main' in 'On Device' column and appear in Device view with 'In Library'. Normality achieved.

But wait, I can make it worse! I can instead do (from the end of the first list above):

1. Eject Kobo without sending book again
2. Wait for Kobo 'import books' to run and new book to appear in Kobo list
3. Reconnect Kobo
4. Book does not shows anything in 'On Device' column. But there is now a book with that name(after plugboard) in the Device view showing nothing for 'In Library'.

Why it matters to me:

Since I'm using KOReader, not the native Kobo app, I'm not running Kobo's import regularly. So new books are on the device, but are invisible to Calibre.


I'm using Calibre 8.4 in Win10 and a new Kobo Clara Colour. I've attached shots of my KoboTouch driver config--I turned off everything I thought would make a difference.

[JSWolf]

If you are running KOReader when you connect to calibre to send eBooks, they won't import when you eject. You would need to go back to Nickel. You can use NickelMenu to force an import of new eBooks. I don't know if you can connect to calibre and then eject to force an import or you have to add at least one eBook.

This is how it worked when I've added eBooks via calibre while running KOReader on my H2O. I've not does this anytime recently or with my Libra 2. One other thing I've never done is to add eBooks via calibre to my Kobo (running Nickel) and not eject from calibre afterwards. So I don't know if you should expect an import when ejecting from Windows. Are you doing a safely remove hardware and eject media or eject via Windows Explorer? If you are doing one, try the other.

[chaley]

The failure after the first "step 6" doesn't surprise me. The import must run to fully put the book into the database. I'm not sure what it does, but I think it sets up a UUID and fills in other tables. When you restart calibre without having run the import, calibre's scan might see "wrong" information and might put that wrong information back into the database.

Speculation: your second set of 5 steps fixes that by resetting the database for the book to what it is when the book is first sent. The import (series 2 step 3) sees the new book and does what it needs to do.

Speculation: the third set fails to match the device because the UUID in the database doesn't match anything in the calibre library. Does using "Match book to library" in the device view fix the problem? Matching the book will update the UUID in the kobo database.

You didn't say whether the "normal" sequence works or not, where "normal" is to skip from section 1 step 4 to section 3 step 1. My guess is that it works, as that is what most people do.

[DNSB]

What you are seeing is expected. Books sent to your Kobo will not show as on device until after they are imported to the Kobo's database. Since quitting calibre and then restarting calibre does not import the books, they are not present in the Kobo database and will not show as on-device.

Since I have my on-device books shared between Nickel, KOReader and Plato, I send the books, safely eject to import the books to the Kobo database and then reconnect to update metadata. This also allows me to use KOReader's search of the calibre.metadata file to work.

[JimmXinu]

Originally Posted by chaley

Speculation: the third set fails to match the device because the UUID in the database doesn't match anything in the calibre library. Does using "Match book to library" in the device view fix the problem? Matching the book will update the UUID in the kobo database.

I've never used that before, so not sure what it's meant to do.

I clicked it, it didn't find the original automatically, but did present the correct book as the only option when I clicked 'match manually'.

Only difference I see after that is the icon in 'In Library' column in Device view changed.

Originally Posted by chaley

You didn't say whether the "normal" sequence works or not, where "normal" is to skip from section 1 step 4 to section 3 step 1. My guess is that it works, as that is what most people do.

Sorry, I left that implicit. Normal Kobo native steps: (send, disconn, kobo imports, reconn) work as expected.

I agree that most people will do that, and I acknowledge I've found an edge case, as far as Kobo native users are concerned.

But it does seem to be a problem with using KOReader. I am (perhaps foolishly) trying to use KOReader while still keeping the ability to fall back to Kobo native.

[JimmXinu]

Originally Posted by DNSB

Since I have my on-device books shared between Nickel, KOReader and Plato, I send the books, safely eject to import the books to the Kobo database and then reconnect to update metadata. This also allows me to use KOReader's search of the calibre.metadata file to work.

Would I be correct in thinking that when you update books you:

1. Exit KOReader back to Kobo native (nickel).
2. Connect Kobo.
3. Send books.
4. Eject Kobo.
5. Wait for book imports.
6. Reconnect Kobo.
7. Have Calibre send more metadata--probably from KoboUtilities.
8. Eject Kobo.
9. Restart KOReader.

Because that's a lot of steps for something I often do several times a day.

[DNSB]

If you want to keep books that you will read with KOReader separate from books that you will use Nickel to read, you could send those books to a hidden directory that is excluded from Nickel's search using the ExcludeSyncFolders in Kobo eReader.conf.

Since I start KOReader and Plato from the Kobo GUI, it's not a real effort for me to safely eject without unplugging the USB cable and then allow import to run before using NickelMenu to trigger a USB connect to allow the metadata update.

Otherwise, you would need to have to create a hidden directory, disable the KoboTouch driver and configure calibre to send to that hidden directory. This would would not allow me to easily compare how books render using Nickel, KOReader or Plato so I've never bothered.

[DNSB]

Originally Posted by JimmXinu

Would I be correct in thinking that when you update books you:

1. Exit KOReader back to Kobo native (nickel).
2. Connect Kobo.
3. Send books.
4. Eject Kobo.
5. Wait for book imports.
6. Reconnect Kobo.
7. Have Calibre send more metadata--probably from KoboUtilities.
8. Eject Kobo.
9. Restart KOReader.

Because that's a lot of steps for something I often do several times a day.

Pretty much correct. Please note that safely eject does not mean that you need to unplug the USB cable. This allows you to trigger a USB connect after the import (if nothing else, it saves wear and tear on the USB connector). You can also set the AutoUSBGadget so you don't get the connect/cancel popup.

[JimmXinu]

Originally Posted by DNSB

Pretty much correct. Please note that safely eject does not mean that you need to unplug the USB cable. This allows you to trigger a USB connect after the import (if nothing else, it saves wear and tear on the USB connector). You can also set the AutoUSBGadget so you don't get the connect/cancel popup.

Agreed. I'm already using NickelMenu and KOReader's 'Start USBMS' for 'reconnect' steps.

I acknowledge my use pattern is likely atypical.

I'm beginning to think that the best approach for me may be to disable KoboTouch and make it a user defined device.

But considering I only vaguely recalled that feature exists, I need to go learn a bit more first.

[JimmXinu]

Initial testing seems to indicate that user defined device is better for me -- in basically all regards.

It solves the problem I was having, and it's even twice as fast doing get / send metadata to device.

Obviously I'm losing all the Kobo native/nickel integration, but in cursory testing nickel seems generally okay with epubs appearing and disappearing from disk when it isn't running. It's purely a backup for me, not something I plan to use, and I can clean the DB some day if necessary.

Anything I've missed in my haste that will bite me later?

[DNSB]

Not off the top of my head. I've known a few people who used KOReader and Plato as their main renderers and went for a drive or user device connection and they seemed happy enough. Oddly, using the KOReader WiFi connection seems to have fewer fans than I expected but that may just be plugging in a USB cable while you are already sitting at the computer using calibre may simply be a better workflow fit.

[JimmXinu]

After my last post, I did try wireless / SmartDevice, just to be thorough.

In some ways it was better:

• Surprising fast for transfers (small ones, I didn't stress test).
• Stays connected in the background on the reader--I could keep reading while transferring books, except for notification pop ups.
• Connect / Disconnect is super quick compared to KOReaders USBMS.

The deal breaker for me is needing to share a wifi.

At home, it's great. But I don't want to expose my Calibre and book reader on a hotel/etc public wifi, and my phone's hotspot feature won't let them talk to each other.

I might look into it more, but my understanding is that Calibre's metadata.calibre and the SmartDevice version saved by the device are not generally compatible. Which prevents using both SmartDevice and USB on the same dir on the device.

[JSWolf]

Originally Posted by DNSB

Pretty much correct. Please note that safely eject does not mean that you need to unplug the USB cable. This allows you to trigger a USB connect after the import (if nothing else, it saves wear and tear on the USB connector). You can also set the AutoUSBGadget so you don't get the connect/cancel popup.

You can use NickelMenu to trigger a USB connection without the need to unplug the USB cable.

[chaley]

Originally Posted by JimmXinu

At home, it's great. But I don't want to expose my Calibre and book reader on a hotel/etc public wifi, and my phone's hotspot feature won't let them talk to each other..

If you have both calibre on a laptop and the reader with you, why not run a hotspot on the laptop? Or easier, get a small travel router that both the laptop and the reader connect to. I've done both.

If the calibre machine is at home then some people report success using tailscale. There is a GitHub project supporting Kobo clients.

[chaley]

Another possibility if your calibre server is on your home net: use a portable router with VPN capability. Assuming you are willing to have a VPN server on your home network open to the wilds, that would permit your reader to connect to the portable router then access calibre using the VPN.