Editing and updating metadata on the device

[JJ Johnson]

I'm trying out Calibre's ability to edit metadata. I have a couple of books where titles are incorrectly capitalized and a couple where author sort strings were incorrect. I've edited them in Calibre, and also updated a few covers.

After editing I selected

Device > Update cached metadata on device

assuming this was how the metadata and covers get updated on the device, but nothing was updated on the device itself. If that's incorrect, what do I need to do next?

The devices is a Kobo Libra, with the recent firmware update.

[davidfor]

No, that is for calibres use, not the devices. Calibre maintains a file called metadata.calibre in the root directory of the device. This is a link between the books on the device and the library. It is used to speed up the matching of books on the device to the library. And to help where there are some issues with that matching. Some ereader apps, such as KOReader, will look at this for metadata, but, I do not know of any dedicated ereader device that does.

How the metadata on an is store for display is different for each of the devices. Most are using some sort of database on the device. The device processes the books when they are first seen and update a database of some sort with the metadata. Kobos does this in a batch when you disconnect the device. Kindles do it in the background. I don't know about the others.

As you list that you have a Kobo Libra H2O, then it will process the book the first time it sees it and extract the metadata. If the book is replaced, and the device deems that the book has change (basically the file size has changed), the device will clear the metadata for the book and process it as a new book. When this happens, you lose the reading status, annotations and collections for the book. That is what happens if you manually put the book on the device. If you change a book and manually put it on the device in the same location and file name, the updated metadata and cover should be seen.

If you use calibre's send-to-device function, there is a lot more that can happen. The KoboTouch driver, can side step some of this. The default configuration for the driver is to send replacements to the device so that the device will not see it as a new book. This means you can fix things like spelling and styles or the cover in a book, and it will not affect the reading status or any annotations. This is an option in the driver and can be turned off. This is the default for historical reasons and because I believe it is the more desirable setting.

The driver can also directly directly update the metadata in the database on the device. This is how it sets the series information for the books (Kobo does not read the series info from the sideloaded books). It can also update all the other metadata the device displays to match any changes made in the calibre library. The series info update is on by default, but the rest is off by default (again, history of the driver). This can happen automatically when the device is connected.

If you use the KoboTouchExtended driver, it inherits the above function from the KoboTouch driver, but, it needs to be configured separately.

In short, if you use calibre and change the metadata in a book and then resend it to a Kobo device, the metadata you see on the device will be updated.

If you don't want the driver to update things automatically, my Kobo Utilities plugin can do it for selected books while the device is connected. And it has finer control over what it updates. And a lot of other functions.

As you mention the author sort, Kobo does not use this at all. It expects the author to be in the format "FN LN", and for multiple authors, "FN1 LN1, FN2 LN2". It works out the sorting from that. It works for most cases, but, it doesn't take into account names like "John von Trapp". That will sort as "Trapp, John von" (which might be right depending on nationality).

[hobnail]

Now that someone who knows what they're talking about has answered your question I'll give you my brute force solution.

I like to reformat and clean up books from project Gutenberg and elsewhere and I use Sigil for that (see the Sigil forum here for download instructions). When I get a book that has messed up author names, etc. I drag it out of Calibre to my desktop, delete it from Calibre, and then open it in Sigil and fix its metadata in Sigil's nice metadata editor, and then drag it back to Calibre.

Hopefully I immediately notice that it needs fixing since my procedure destroys the tags and whatnot that I added in Calibre.

[JJ Johnson]

Thanks. That's a lot of helpful information.

Resend books to the device.

Ok, now I have two of many books on the device. I probably shouldn't have resent every book in the Calibre library (I'm lazy), but apart from that, I'm guessing the metadata that I've changed is the very thing used to identify books in the database.

Edit: Actually, I think it may be because of the path of the book changes according to its title and author sort strings. Anyway, I figured out how to use Calibre to remove duplicate books from the device and think I have it mostly solved.

[davidfor]

Originally Posted by JJ Johnson

Thanks. That's a lot of helpful information.

Resend books to the device.

Ok, now I have two of many books on the device. I probably shouldn't have resent every book in the Calibre library (I'm lazy), but apart from that, I'm guessing the metadata that I've changed is the very thing used to identify books in the database.

Edit: Actually, I think it may be because of the path of the book changes according to its title and author sort strings. Anyway, I figured out how to use Calibre to remove duplicate books from the device and think I have it mostly solved.

Yes, that is correct. The save template in the driver is usually based on the title and author. And if you fix these after the first time you send the books to the device, it doesn't replace the original copy. I haven't come up with a good solution to this. It is rarely a problem for me as I make sure the title and author are correct when I add the book to calibre.

[davidfor]

Originally Posted by hobnail

Now that someone who knows what they're talking about has answered your question I'll give you my brute force solution.

I like to reformat and clean up books from project Gutenberg and elsewhere and I use Sigil for that (see the Sigil forum here for download instructions). When I get a book that has messed up author names, etc. I drag it out of Calibre to my desktop, delete it from Calibre, and then open it in Sigil and fix its metadata in Sigil's nice metadata editor, and then drag it back to Calibre.

Hopefully I immediately notice that it needs fixing since my procedure destroys the tags and whatnot that I added in Calibre.

Why not fix the metadata in the calibre metadata editor and then use the Embed metadata tool or Polish books to update the book? And you can always edit a book with Sigil from calibre with the Open with plugin.

[theducks]

Originally Posted by JJ Johnson

Thanks. That's a lot of helpful information.

Resend books to the device.

Ok, now I have two of many books on the device. I probably shouldn't have resent every book in the Calibre library (I'm lazy), but apart from that, I'm guessing the metadata that I've changed is the very thing used to identify books in the database.

Edit: Actually, I think it may be because of the path of the book changes according to its title and author sort strings. Anyway, I figured out how to use Calibre to remove duplicate books from the device and think I have it mostly solved.

IMHO you have probably made a mess of metadata.calibre ON THE DEVICE , BEFORE STARTING CALIBRE, Delete it be prepared for a device rescan (that can take some time on devices with large storage.) , now connect to calibre and grab some coffee and snacks

My way:
Switch to device view: Click the Date column header (your list is now sorted bu date it was added. a second click reverses the direction)

If you had not changed the template (much), a second sort would be to then click Title column
Sorts work slightly backwards from what you might think. What you are doing is applying a Bias to the next sort here. Title in Date added order.

Here is another case. 2 books with the same title. Click Authors, then click Title.. They (the Titles) are now in Authored order

[hobnail]

Originally Posted by davidfor

Why not fix the metadata in the calibre metadata editor and then use the Embed metadata tool or Polish books to update the book? And you can always edit a book with Sigil from calibre with the Open with plugin.

I'll try that next time.

[mallomar]

Originally Posted by davidfor

[...]Kindles do it in the background. I don't know about the others. [...]

If you use calibre's send-to-device function, there is a lot more that can happen. The KoboTouch driver, can side step some of this. The default configuration for the driver is to send replacements to the device so that the device will not see it as a new book. This means you can fix things like spelling and styles or the cover in a book, and it will not affect the reading status or any annotations. This is an option in the driver and can be turned off. This is the default for historical reasons and because I believe it is the more desirable setting.

Hope it's okay to dig up an old thread. My question is very similar and you look like you have expertise in this, so I thought this would be the best place to ask. I have a book I am reading now whose cover I updated on calibre. I would like to have the updated cover on my Kindle without erasing my annotations and highlights for the book, as I am already a good ways through it. Do you know if I can do it on my Kindle the same way you suggested to the OP that he or she can do with their Kobo, by using the send-to-device function again?

[davidfor]

Originally Posted by mallomar

Hope it's okay to dig up an old thread. My question is very similar and you look like you have expertise in this, so I thought this would be the best place to ask. I have a book I am reading now whose cover I updated on calibre. I would like to have the updated cover on my Kindle without erasing my annotations and highlights for the book, as I am already a good ways through it. Do you know if I can do it on my Kindle the same way you suggested to the OP that he or she can do with their Kobo, by using the send-to-device function again?

Sorry, I don't really know. My expectation would be yes, but, I don't use Kindles, so I don't know the details.

[mallomar]

Originally Posted by davidfor

Sorry, I don't really know. My expectation would be yes, but, I don't use Kindles, so I don't know the details.

So, I played around with it a bit. Using Send to device in Calibre, Calibre sent a second copy of the book to the device BUT it updated the cover for both. I deleted the new copy and the new cover remains on the old copy, along with all of the annotations, bookmarks, etc. So, this is a good work around. If anyone else finds this looking for a solution, just remember that if the cover disappears at first you need to disconnect the Kindle and reconnect to Calibre, due to a longstanding Kindle bug. Thanks for the initial guidance!

[JJ Johnson]

Someone remind me again... How do I simply update a book on the device when I change metadata? Emphasis on "simply". I've read through the above again, and it's not making much sense.

Typically, I'll update a cover, fix an author name or book title. I either keep getting duplicate books or else the metadata never updates on the device.

[davidfor]

Originally Posted by JJ Johnson

Someone remind me again... How do I simply update a book on the device when I change metadata? Emphasis on "simply". I've read through the above again, and it's not making much sense.

Typically, I'll update a cover, fix an author name or book title. I either keep getting duplicate books or else the metadata never updates on the device.

If all you are updating is the metadata, such as the title, comments or series info, then the driver will handle it. There is configuration in the driver to do this. The default option is to only update the series info, but the rest of the metadata the Kobo devices can display can be updated. To set the option, you need to open the driver configuration and change the options on the third tab. The simplest way to get correct driver is to connect the device and right-click the device button and choose the option to configure the device. Otherwise, you need to go through the preferences. If you do this and it is still not happening, check that on the "Sending books to devices" page of the preferences that "Metadata management" is set to the automatic option. With this done, calibre will update the device each time it is connected with any changes made.

If you are changing the cover, you can use my Kobo Utilities plugin to send the changed cover to the device. But, the cover won't be in the book when you open it.

To change the book, you should be able to just send it from calibre. The driver does cheat. If you manually changed the book, the device would detect this and remove the original book from its database and add the replacement book as a new book. The driver can prevent this so that you do not lose your reading status or annotations. This will be safe if the changes are to styles or fixing text. If the internal structure of the book changes, it can cause problems, but, that can also be fixed.

If when you send replacement book to the device, you get another copy, it is probably because you changed something in the metadata that is used in the save template that generates the file name for the book on the device. The usual save template has the title and author in it, so changing either of those will result in a duplicate book. I don't have a good solution for that. The device has no issue with multiple copies of the same book. When I need to do this, I send the new version and remove the old version. If it is a book I am reading or have read, I let the device import it, connect again and restore the stored reading status.