Finding books by metadata

[Frenzie]

A proof of concept metadata search shouldn't be too hard to whip up, on the basis of some simple stuff along these lines:

SELECT * FROM bookinfo WHERE filename REGEXP 'README.*'

The main problem is gathering the metadata and keeping it up to date. One time I was unable to use my H2O for *at least* 10 minutes after I unplugged its USB. That may be okay if you purposefully initiate a full scan, but otherwise…

Also see https://github.com/koreader/koreader/issues/6922

Edit: GLOB is supported out of the box, forget REGEXP.

[mst]

The main problem is gathering the metadata and keeping it up to date. One time I was unable to use my H2O for *at least* 10 minutes after I unplugged its USB. That may be okay if you purposefully initiate a full scan, but otherwise…

I haven't experienced that with KoReader yet. In contrast to KoReader, Nickel seems to import the metadata after unmounting the device, so whenever I put a new epub on the device via USB, it imports that metadata and makes it available for search. For one epub, this takes seconds on my Kobo Clara HD. Recently I wiped the memory and put more than 2,000 new files on it. Took about 10 minutes to import.

In KoReader, the "extract and cache book information" produces all the metadata necessary for a useful search function, and it's odd the metadata is unavailable for search after extracting it, only for displaying the files in a list.

For a useful search function, a database with the book title, description, author would already be enough. No need to keep that up to date all the time, only when a file is replaced. Some kind of "speed up book information database" function could help delete orphaned metadata, for instance, to remove the metadata of those files that no longer exist on the device.

[mst]

Because Nobody Commited Code To Support this feature !!!

Understood. This feature would put KoReader over the edge and make Nickel pale in comparison, IMO. Seems to me most of the coding work is already done, because the "extract and cache book information" feature already exists and saves that information somewhere but stops short of making it available for search.

Caching just the full book title, author, and description would suffice and speed up the metadata gathering process.

[Frenzie]

Originally Posted by mst

For a useful search function, a database with the book title, description, author would already be enough. No need to keep that up to date all the time, only when a file is replaced. Some kind of "speed up book information database" function could help delete orphaned metadata, for instance, to remove the metadata of those files that no longer exist on the device.

All of that functionality already exists. In the file browser there's Display mode → Book info cache management. Also in the file browser, in the plus menu there's Extract and cache book information.

What doesn't exist yet is a GUI to facilitate queries like in the attached screenshot.

[mst]

All of that functionality already exists. [...] What doesn't exist yet is a GUI to facilitate queries like in the attached screenshot.

That's great to know. I for one find a function to query metadata much more useful than the file search function.

Now, editing the metadata directly on the Kobo is probably more involved and another story.

[Alan_S]

Originally Posted by Frenzie

The main problem is gathering the metadata and keeping it up to date. One time I was unable to use my H2O for *at least* 10 minutes after I unplugged its USB. That may be okay if you purposefully initiate a full scan, but otherwise…

Exactly the same experience on my side. One of (many) reasons why I don't want to have anything with nickel. And, why I store my ebooks, just to be safe, in hidden folders.

I really don't understand people who like to wait and wait and wait...

[Galunid]

You can use this script to organize your books into directories based on tags https://pastebin.com/tPGTcjYc. You need to modify *hierarchy* variable to fit your liking, create new "Column build from columns" in calibre. Enter the resulting "program" from the script into the "Template" field (of the custom column). When you connect your device to calibre click "Configure this device" -> Save Template -> {#lookup name} of your column.

You can see more on how it works and what it does in.
https://manual.calibre-ebook.com/tem...custom-columns
https://manual.calibre-ebook.com/tem...l-program-mode

[mst]

The main problem is gathering the metadata and keeping it up to date. One time I was unable to use my H2O for *at least* 10 minutes after I unplugged its USB. That may be okay if you purposefully initiate a full scan, but otherwise…
Exactly the same experience on my side. One of (many) reasons why I don't want to have anything with nickel. And, why I store my ebooks, just to be safe, in hidden folders.

I really don't understand people who like to wait and wait and wait...

I experience this after putting books on the device while using KoReader and then switching back to Nickel. KoReader doesn't scan after disconnecting from USB, so we get used to it. When you are in Nickel, plug in and then unplug the device, it will scan all new books, even though we might not have put any new files on the device while using Nickel. When you reshuffled your library in KoReader this scan can take a while. On the other hand, gathering metadata takes a long time in KoReader as well...

Of course I prefer KoReader but have to switch into Nickel once in a while because of the missing metadata search functionality in KoReader, so that scan and wait time is a necessary evil, unfortunately.

[mst]

Apologies in advance, it seems I'm missing something, but I still have trouble finding my epubs in KoReader on my Kobo Clara HD by Calibre bookshelves.

Could a patient soul walk me through the process of putting books from Calibre to my device so I can browse them by bookshelves? Would greatly appreciate this.

[pazos]

The calibre plugin in KO has 3 options:

- Find books
- Browse series
- Browse tags

Find books will browse them by title, authors or paths based on your configuration in the plugin.

Browse tags / series will do the same with their respective calibre columns.

In all cases you can introduce a search value to grep the results or leave it empty to get all the results for a given search.

Some examples:

- Find books: "" -> All books
- Find books: "pattern" -> Books which pattern in title, authors or path.
- Browse series: "" -> All series
- Browse series: "pattern" -> Series that match the pattern
- Browse tags: "" -> All tags
- Browse tags: "pattern" -> Tags that match the pattern

There's no hidden behaviour. Just these three options.

[mst]

Thank you, Pazos. It's still unclear, though.

To put a book on the device, I connect it via USB and then click "send to device" in Calibre. When I do this with only one book, I see a folder in the root of the device with the author's name and the epub file in it. Each book gets its own folder in the root of the device. Still, I can't browse tags properly.

How should I "configure the device" in Calibre so that the tags transfer to the device?

How should I set up "collections" or bookshelves so I can browse by bookshelves I created in Calibre?

What can I do to only see the tags and bookshelves that I created in Calibre? Because still, I see a wild jumble of tags when I try to browse by tags on KoReader.

[pazos]

Fix your calibre tags first. Choose a single book. Adds the tags you want in the calibre TAGS column. Send the book to the device.

Go to find by books, insert the name of the book. Check that you can see the book. Long press the book: check that the info displayed by the book is the same as you see in calibre.

Good, now repeat for your entire library. Remember only a few calibre metadata is used. No custom columns.

[mst]

Thanks a lot, Pazos! I see the tags now, and the other tags have disappeared. It's a miracle!

The next step are the bookshelves, because they are the main organizing principle in my library on Calibre. I have bookshelves like "Fiction", "Biographies", "History", etc. I see you just created them as tags. I created them as series now, so they should be available on the device. However, I now see the tags, but not the series, even though I enabled "set series information" in the device configuration. Do I need to refresh some cache before I see the series information also?

Really appreciate your help and patience.

[NiLuJe]

"Bookshelves" are a Nickel concept. There's no direct equivalent.

On the other hand, you must be creating your Bookshelves out of *something* in Calibre, and that something is usually another piece of metadata than we *can* work with.

Since you mentioned tags, that works, as long as you're disciplined in how your Calibre library is tagged, so you're not drowned in a sea of useless tags.

(e.g., back when I actually bothered about Collections (the Kindle's equivalent of Nickel bookshelves), I based them on a set of very specific tags, which where prefixed with a specific character. I still keep those to this day to group books by genre, so I have tags like +Fantasy, +Mystery, etc. And those pop up at the top of our "Browse by tags" results thanks to the magic of alphabetical sorting ).

I've never actually used Bookshelves on Kobo though, because, in practice, I find that they just bog down both Nickel & Kindle, and a plain Search by other metadata is just much faster. (Which means I can't tell whether my "tags with a prefix" idea works with the Kobo driver. It worked on Kindle because the Kindle Collections plugin was very specifically engineered for that kind of trickery).

[mst]

The +XYZ tags are a great idea, many thanks!

I made a custom category in Calibre that I called "bookshelves" and then forget that it was custom. I am converting this to series now and see how that works.