[Kobo] calibre does not find books in subsubfolders of device

[avv]

On my Kobo Libra H2O I am running KOreader.

Since I do not want to clutter the main folder I created a folder called "Books" and send my Calibre Library to this folder by editing the "save template" of the device. This works like a charm.

The problem is that the next time I connect my device, Calibre thinks my device is empty, I guess because it does not scan the subsubfolders of the main memory ? This is annoying because I cannot see which books are already on the device, update metadata etc.

I have looked through all of the settings and cannot find a way to fix this problem. Any suggestions ? Thanks!

[PeterT]

Remember that calibre uses the Kobo managed database on the device to list books, so unless nickel has processed your added books, calibre will not list them.

Sent from my Pixel 4a using Tapatalk

[avv]

Originally Posted by PeterT

Remember that calibre uses the Kobo managed database on the device to list books, so unless nickel has processed your added books, calibre will not list them.

Sent from my Pixel 4a using Tapatalk

aaaah I did not consider this. How do I get nickel to "process my added books" ? Thanks !

[DNSB]

Originally Posted by avv

aaaah I did not consider this. How do I get nickel to "process my added books" ? Thanks !

You could use NickelMenu and it's Import Books option. Or connect and disconnect from your PC while running Nickel. I'm assuming your Books directory is in the root of the Kobo exposed storage and does not have a leading . as part of the directory name.

[davidfor]

@avv: If you only use KOReader, then you should not use the KoboTouch or KoboTouchExtended drivers. I recommend using either the Folder Device Interface or the User Defined USB driver. With the former, you connect to a folder on the device. With the latter, you use the USB ids from the device and calibre will detect it when you plug the device in. Both of these will scan the device for books using the files system rather than using the database.

[avv]

Originally Posted by DNSB

You could use NickelMenu and it's Import Books option. Or connect and disconnect from your PC while running Nickel. I'm assuming your Books directory is in the root of the Kobo exposed storage and does not have a leading . as part of the directory name.

I tried this, now the nickel interface has changed a lot and I cannot find how to go back into Koreader ?? There is a tab NickelMenu that has only one option: "generator error". When I click it it says: "kfmon: error checking "/tmp/kfmon-icp-ctl":stat: No suck filde or directory (scr/generator_c.c:93)."

How can I fix this ?

[avv]

Originally Posted by avv

I tried this, now the nickel interface has changed a lot and I cannot find how to go back into Koreader ?? There is a tab NickelMenu that has only one option: "generator error". When I click it it says: "kfmon: error checking "/tmp/kfmon-icp-ctl":stat: No suck filde or directory (scr/generator_c.c:93)."

How can I fix this ?

Okay I fixed it via a factory reset. I will take davidfor's advice and connect to the mounted folder instead of connecting to the kobo device. Thank you!

[DNSB]

Originally Posted by avv

I tried this, now the nickel interface has changed a lot and I cannot find how to go back into Koreader ?? There is a tab NickelMenu that has only one option: "generator error". When I click it it says: "kfmon: error checking "/tmp/kfmon-icp-ctl":stat: No suck filde or directory (scr/generator_c.c:93)."

How can I fix this ?

The basic NickelMenu interface has none of the options enabled. If you use the One-Click Install Package install, it will give you one or two options, depending on whether you installed one of KOReader or Plato or both.

If you use the NickelMenu install, a sample options file is in the .adds/nm directory called doc. You can make a copy of this file and edit it to enable various options by uncommenting them (remove the #s at the line starts) and/or adding other options found in the NickelMenu thread.

[vietchovui]

Originally Posted by PeterT

Remember that calibre uses the Kobo managed database on the device to list books, so unless nickel has processed your added books, calibre will not list them.

Sent from my Pixel 4a using Tapatalk

Most of books on my Forma are sideloaded via GDrive and listed on nickel (Books menu), but Calibre cannot see them. It shows only books transfer via "send to device". Is there a way to show all the books on kobo (load from GDrive and other ways) in calibre?

[davidfor]

Originally Posted by vietchovui

Most of books on my Forma are sideloaded via GDrive and listed on nickel (Books menu), but Calibre cannot see them. It shows only books transfer via "send to device". Is there a way to show all the books on kobo (load from GDrive and other ways) in calibre?

Any book that a Kobo device shows in the book lists should be seen by calibre. The KoboTouch driver reads the database on the device to find books. But, they might not automatically match to books in your calibre library. Look in the device list (press the device button to see it), and check for the books there. Books that match to a book in your library will have a green tick. For books that do not have a green tick, calibre was not able to find the book in the library. You can manually match the books from the list.

If there are books that are not shown in the device list at all then there is a problem. Please run calibre in debug mode and post the logs from connecting the device. Hopefully I will be able to see why the books have not been found.

[vietchovui]

Originally Posted by davidfor

If there are books that are not shown in the device list at all then there is a problem. Please run calibre in debug mode and post the logs from connecting the device. Hopefully I will be able to see why the books have not been found.

I have tried to logout my kobo account and login again to rebuild the kobo database (about 90 books are always in kobo storage). But I still get missing books when connecting kobo to calibre (via USB cable or koreader calibre wireless). Hope that you could find a solution based on the log I attach here.
Thank you,

[davidfor]

Originally Posted by vietchovui

I have tried to logout my kobo account and login again to rebuild the kobo database (about 90 books are always in kobo storage). But I still get missing books when connecting kobo to calibre (via USB cable or koreader calibre wireless). Hope that you could find a solution based on the log I attach here.
Thank you,

Firstly, I can't help with KOReader and the Wireless driver. I don't use either and have no real idea how hey work. But, there is an error in the log which probably explains why there are no books. Also, the book list shown by KOReader and then this driver has no direct relationship with the books shown by the Kobo software (nickel) and either the KoboTouch or KoboTouchExtended drivers. Nickel and KOReader find books in completely different ways and may not be searching the same places on the device.

The results using the KoboTouchExtended driver is very strange. It appears to have only found three book, but, the device list shows five. And most of those are screenshots. Unfortunately, there is not enough information in the debug log. Which again is unusual. One of the few things I can think of is that all your books are previews. The default configuration for the driver is to not show previews. This can be changed and these books will appear.

Can you do some things and questions to answer:

Take a screen shot of the book list in nickel. I want to see more details of the books as the device sees them.

In the driver, set the "Title to test when debugging" to the name of one of the books that is not showing. Then run calibre in debug mode with the device connected and post the log again. This option will show details about the book in the debug log. You do not need to put the full book title in, just the start. But, it must be an exact match to the title.

Send a book from calibre to the device. When you disconnect, check that the book is imported and displayed. Then connect again see if calibre shows the book in the device list and matched to the book in the library.

What happens when you disconnect the device? Is it showing the books correctly? Does it reimport any sideloaded books? Do you lose any reading status?

[vietchovui]

Originally Posted by davidfor

The results using the KoboTouchExtended driver is very strange. It appears to have only found three book, but, the device list shows five. And most of those are screenshots.

Sorry, this is my mistake. The PC screenshot (with 6 books) was not made at the same time (after adding 3 screenshots on kobo) as the debug log (at the time there were only 3 books).

Originally Posted by davidfor

The default configuration for the driver is to not show previews. This can be changed and these books will appear.

I've never heard about this before. When I rechecked, it was in default state (not checked - disable)

Originally Posted by davidfor

Can you do some things and questions to answer:

I include the log and some screenshots in this post.
The KoboTouchExtended settings:
- Show previews is checked
- Title to test when debugging: Anh em (you can see there is a book with the same beginning words in the screenshot)

Originally Posted by davidfor

Send a book from calibre to the device. When you disconnect, check that the book is imported and displayed. Then connect again see if calibre shows the book in the device list and matched to the book in the library.
What happens when you disconnect the device? Is it showing the books correctly? Does it reimport any sideloaded books? Do you lose any reading status?

Every things work fine ONLY for the NEWLY added book.

[vietchovui]

Finally I figured it out. It is not a calibre or kobo bug. After reading through many posts on this forum, I know that calibre doesn't read database created by kobo (nickel), but creates its own database on kobo storage. So books are not sideloaded by calibre but by other ways (copy paste or via kobo cloud services in my case) will not be recognized by calibre. Thank you for your patient help @davidfor.

[davidfor]

Originally Posted by vietchovui

Finally I figured it out. It is not a calibre or kobo bug. After reading through many posts on this forum, I know that calibre doesn't read database created by kobo (nickel), but creates its own database on kobo storage. So books are not sideloaded by calibre but by other ways (copy paste or via kobo cloud services in my case) will not be recognized by calibre. Thank you for your patient help @davidfor.

Where on earth do you get that idea from? Not from the person who maintains the driver, especially as I have already explicitly stated that calibre reads the database on the device.

Calibre does maintain a file on the device. That is used to make it faster to load the books and to help where matching the metadata between the book on the device and in the library does not work for some reason. Removing this file, doesn't really harm anything.

If these books are being loaded by the Kobo Cloud service, and calibre cannot find them, then it means that Kobo Cloud is doing something. And looking at the code, it appears to be faking an SD card and putting the books on that. Luckily for them, it fools nickel into loading books from it, but, doesn't fool nickel enough to for when the device is connected to a PC. If you look at the details of one of the books in nickel, the file name will probably start with "/mnt/sd". If so, that is the reason the books are not being found.

There are ways they could do this without faking the SD card. There are other ways to get nickel to look for new books that would be compatible with the drivers in calibre.