sideloaded epubs showing in Nickel, not KOReader?

Uncle Robin · 07-13-2021, 04:16 AM

My Libra is running the current firmware and KOReader tells me it's up to date "v2021.06.30gbfda4bc_2021-07-10"

I added 3 books (sideloaded epubs) to my device, and NONE of them are showing up in KOReader at all.

At first I thought it was just the series info and tags that weren't showing up, so I deleted calibre.metadata, reconnected my device, and first selected "update cached metadata on device" then ran Kobo Utilities "Update metadata on deice twice, unplugging and replugging between each run. NO metadata changed in KOReader, including the 35 or so books that should have had the "TBR" tag removed.

That got me thinking, so I checked for the 3 books by the authors' names, and they are not showing up in KOReader. They ARE on the device, all show up in Nickel, and one is open there, but they are invisible to KOReader it seems.

How can I fix these problems? I would (a) love to read the books in KOReader and (b) have metadata update on device when I synch with Calibre.

Uncle Robin · 07-13-2021, 04:39 AM

UPDATE: I tried removing the books from my device, deleting and re-creating calibre.metadata, refreshing all metadata on device, then sending them to my device again. No change - visible in Nickel, not in KOReader, and none of the metadata updates are showing in KOReader either

Frenzie · 07-13-2021, 04:42 AM

KOReader doesn't integrate with Nickel metadata, so nothing would be expected to change in that regard. If you click on the plus icon in the top right you can "choose extract and cache metadata" although I'm not sure otoh if that also updates.

What are the names of the files you copied onto the device? Under file browser settings (leftmost icon in the top menu → settings) you can enable show hidden files and show unsupported files.

Uncle Robin · 07-13-2021, 04:53 AM

Quote:

Originally Posted by Frenzie

KOReader doesn't integrate with Nickel metadata, so nothing would be expected to change in that regard. If you click on the plus icon in the top right you can "choose extract and cache metadata" although I'm not sure otoh if that also updates.

What are the names of the files you copied onto the device? Under file browser settings (leftmost icon in the top menu → settings) you can enable show hidden files and show unsupported files.

It's not Nickel metadata I was trying to read. The series info, tags etc, all come from Calibre, and KOReader normally (though not always) updates its metadata listings when I update Calibre's and synch the device.

I decided to check the filename of the epub so connected the device to my PC from KOReader, not Calibre, and took note of the filename. When I ejected the device, I searched the directory structure, and found the epub and opened it successfully - it was not a hidden file, and it was not an unsupported filetype, just a stock epub. But it still doesn't show up when I do a Calibre metadata search "Browse series", "browse tags", "search books" - all come up empty.

One curiosity I did notice was that there are now 17 books by this author on my device, and KOReader finds the other 16 via a metadata search. The one it doesn't find is the only epub, the others are all kepubs. That struck me as curious.

Frenzie · 07-13-2021, 06:07 AM

That sounds like the calibre metadata may not have been transferred correctly, but I don't know much about any of that.

Uncle Robin · 07-13-2021, 06:16 AM

Quote:

Originally Posted by Frenzie

That sounds like the calibre metadata may not have been transferred correctly, but I don't know much about any of that.

I added 3 books yesterday, and all updated just fine. It is odd.

pazos · 07-13-2021, 11:43 AM

The calibre plugin is pretty dumb. All it does is reading the contents of "calibre.metadata" (or "metadata.calibre", I don't remember)

Most user just have one calibre library on their ereaders. They often use the platform driver for calibre. In the case on Kobos the rootdir of the calibre library is in /mnt/onboard.

Please check that this file contains info about the book you transferred. If that's true it is the plugin fault. If that file doesn't contain info about that specific book then the plugin works as expected.

Uncle Robin · 07-13-2021, 02:55 PM

Quote:

Originally Posted by pazos

The calibre plugin is pretty dumb. All it does is reading the contents of "calibre.metadata" (or "metadata.calibre", I don't remember)

Most user just have one calibre library on their ereaders. They often use the platform driver for calibre. In the case on Kobos the rootdir of the calibre library is in /mnt/onboard.

Please check that this file contains info about the book you transferred. If that's true it is the plugin fault. If that file doesn't contain info about that specific book then the plugin works as expected.

I opened metadata.calibre in Notepad++ and the three "missing" books are all showing up in that file, as in the attached screenshot.

NiLuJe · 07-13-2021, 03:02 PM

See if perhaps https://www.mobileread.com/forums/sh...d.php?t=339947 doesn't apply to your current predicament

.

It appears that you're doing the reverse, but one can never be too sure

.

And to expand on what @pazos said, do you see an lpath entry for those books in the metadata file?

Uncle Robin · 07-13-2021, 07:15 PM

Quote:

Originally Posted by NiLuJe

See if perhaps https://www.mobileread.com/forums/sh...d.php?t=339947 doesn't apply to your current predicament

.

It appears that you're doing the reverse, but one can never be too sure

.

And to expand on what @pazos said, do you see an lpath entry for those books in the metadata file?

Reading that linked thread, scenario 2 fits exactly what I'm doing

Quote:

Scenario 2: Sending book from calibre to Kobo via USB when nickel is open

Exit koreader
Nickel is running
Connect USB
Send new book from Calibre to main memory
Check file on root of kobo: metadata.calibre: The new book gets an new entry there
Eject Kobo from Calibre
Nickel shows import dialog
Open Koreder
Go to file explorer: New book is there as a file
Connect Kobo again via USB

OK: Check metadata.calibre: The entry for the new book is still there.
OK: In Calibre the book is shown as existing on device in library (Green check mark)

EXCEPT that the books are not showing up in KOReader when I do a Calibre metadata search. A check confirmed that they do all have a "path" entry in metadata.calibre

Uncle Robin · 07-13-2021, 07:57 PM

Re-reading that linked thread, I decided to try to force a Nickel scan. Having not the foggiest idea how to do this, I connected my Libra by USB in Nickel, without Calibre running. I then ejected it, scrolled through my entire list of books, and then opened the book I'm currently reading (one of the 3 not showing up in KOReader's Calibre metadata search). Then, I reconnected to my PC, still in Nickel, still without Calibre running. After a few minutes I ejected, fired up KOReader and hey presto! - the missing books are now all showing up.

The reason I'm posting this here is because the earlier thread mentioned chaining USBMS sessions from inside KOReader. In the 5-6 months I've been running KOReader, I've probably run fewer than 10 USBMS sessions in total from inside KOReader before running into this problem, so it was not the cause of my issue, I think.

pazos · 07-13-2021, 08:36 PM

@Uncle Robin: try to disable the cache in the plugin settings. Can you reproduce without cache?.

Uncle Robin · 07-13-2021, 08:46 PM

Quote:

Originally Posted by pazos

@Uncle Robin: try to disable the cache in the plugin settings. Can you reproduce without cache?.

Thanks for that. I will try that later, but before I do, I just want to make 100% sure I understand correctly: You are asking me to UNcheck the highlighted option, and see if the problem (which has gone away at the moment) returns?

pazos · 07-14-2021, 04:53 PM

Quote:

Originally Posted by Uncle Robin

Thanks for that. I will try that later, but before I do, I just want to make 100% sure I understand correctly: You are asking me to UNcheck the highlighted option, and see if the problem (which has gone away at the moment) returns?

Nope. Sorry. I missed the part where you said you solved your issue. If you find the same issue again you can test if you're able to reproduce without cache.

The cache is regenerated when the modification time of "metadata.calibre" is newer than the cache file itself. That won't happen if somewhat the file is modified without changing mtime.

Anyways, you don't need to really disable the cache even if that happens. You just need to delete the file

Code:

/mnt/onboard/.adds/koreader/cache/calibre/books.dat

and the cache will be generated from the current file on the next run.

Still clueless about the root cause of your issue, but that shouldn't be platform dependant (from the PoV of KOReader, dunno much about calibre device drivers)

Uncle Robin · 07-14-2021, 10:14 PM

Quote:

Originally Posted by pazos

Anyways, you don't need to really disable the cache even if that happens. You just need to delete the file

Code:

/mnt/onboard/.adds/koreader/cache/calibre/books.dat

and the cache will be generated from the current file on the next run.

Still clueless about the root cause of your issue, but that shouldn't be platform dependant (from the PoV of KOReader, dunno much about calibre device drivers)

Thanks for the pointer on which file to delete if the problem reoccurs, very helpful!

07-13-2021, 04:16 AM	#1
Uncle Robin Diligent dilettante Posts: 3,661 Karma: 52758936 Join Date: Sep 2019 Location: in my mind Device: Kobo Sage; Kobo Libra Colour	sideloaded epubs showing in Nickel, not KOReader? My Libra is running the current firmware and KOReader tells me it's up to date "v2021.06.30gbfda4bc_2021-07-10" I added 3 books (sideloaded epubs) to my device, and NONE of them are showing up in KOReader at all. At first I thought it was just the series info and tags that weren't showing up, so I deleted calibre.metadata, reconnected my device, and first selected "update cached metadata on device" then ran Kobo Utilities "Update metadata on deice twice, unplugging and replugging between each run. NO metadata changed in KOReader, including the 35 or so books that should have had the "TBR" tag removed. That got me thinking, so I checked for the 3 books by the authors' names, and they are not showing up in KOReader. They ARE on the device, all show up in Nickel, and one is open there, but they are invisible to KOReader it seems. How can I fix these problems? I would (a) love to read the books in KOReader and (b) have metadata update on device when I synch with Calibre.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Koreader/Nickel Shared Folders	jbenwell	KOReader	17	06-09-2021 11:35 AM
KOReader vs patched Nickel?	Uncle Robin	KOReader	26	01-16-2021 06:59 AM
Not able to select Koreader via Nickel...?	Revan	KOReader	12	01-29-2018 11:06 PM
Aura H2O Koreader and malformed nickel database	Patricius	Kobo Reader	38	08-03-2015 05:45 AM

07-13-2021, 04:39 AM	#2
Uncle Robin Diligent dilettante Posts: 3,661 Karma: 52758936 Join Date: Sep 2019 Location: in my mind Device: Kobo Sage; Kobo Libra Colour	UPDATE: I tried removing the books from my device, deleting and re-creating calibre.metadata, refreshing all metadata on device, then sending them to my device again. No change - visible in Nickel, not in KOReader, and none of the metadata updates are showing in KOReader either

07-13-2021, 04:42 AM	#3
Frenzie Wizard Posts: 1,742 Karma: 730681 Join Date: Oct 2014 Location: Antwerp Device: Kobo Aura H2O	KOReader doesn't integrate with Nickel metadata, so nothing would be expected to change in that regard. If you click on the plus icon in the top right you can "choose extract and cache metadata" although I'm not sure otoh if that also updates. What are the names of the files you copied onto the device? Under file browser settings (leftmost icon in the top menu → settings) you can enable show hidden files and show unsupported files.

07-13-2021, 06:07 AM	#5
Frenzie Wizard Posts: 1,742 Karma: 730681 Join Date: Oct 2014 Location: Antwerp Device: Kobo Aura H2O	That sounds like the calibre metadata may not have been transferred correctly, but I don't know much about any of that.

07-13-2021, 11:43 AM	#7
pazos cosiñeiro Posts: 1,406 Karma: 2451781 Join Date: Apr 2014 Device: BQ Cervantes 4	The calibre plugin is pretty dumb. All it does is reading the contents of "calibre.metadata" (or "metadata.calibre", I don't remember) Most user just have one calibre library on their ereaders. They often use the platform driver for calibre. In the case on Kobos the rootdir of the calibre library is in /mnt/onboard. Please check that this file contains info about the book you transferred. If that's true it is the plugin fault. If that file doesn't contain info about that specific book then the plugin works as expected.

07-13-2021, 03:02 PM	#9
NiLuJe BLAM! Posts: 13,506 Karma: 26047190 Join Date: Jun 2010 Location: Paris, France Device: Kindle 2i, 3g, 4, 5w, PW, PW2, PW5; Kobo H2O, Forma, Elipsa, Sage, C2E	See if perhaps https://www.mobileread.com/forums/sh...d.php?t=339947 doesn't apply to your current predicament . It appears that you're doing the reverse, but one can never be too sure . And to expand on what @pazos said, do you see an lpath entry for those books in the metadata file?

07-13-2021, 07:57 PM	#11
Uncle Robin Diligent dilettante Posts: 3,661 Karma: 52758936 Join Date: Sep 2019 Location: in my mind Device: Kobo Sage; Kobo Libra Colour	Re-reading that linked thread, I decided to try to force a Nickel scan. Having not the foggiest idea how to do this, I connected my Libra by USB in Nickel, without Calibre running. I then ejected it, scrolled through my entire list of books, and then opened the book I'm currently reading (one of the 3 not showing up in KOReader's Calibre metadata search). Then, I reconnected to my PC, still in Nickel, still without Calibre running. After a few minutes I ejected, fired up KOReader and hey presto! - the missing books are now all showing up. The reason I'm posting this here is because the earlier thread mentioned chaining USBMS sessions from inside KOReader. In the 5-6 months I've been running KOReader, I've probably run fewer than 10 USBMS sessions in total from inside KOReader before running into this problem, so it was not the cause of my issue, I think.

07-13-2021, 08:36 PM	#12
pazos cosiñeiro Posts: 1,406 Karma: 2451781 Join Date: Apr 2014 Device: BQ Cervantes 4	@Uncle Robin: try to disable the cache in the plugin settings. Can you reproduce without cache?.