New Calibre feature

[tumbi]

The current Calibre release includes the following new feature:

“Kobo driver: Add an option to directly update metadata in the Kobo device database, instead of waiting for the Kobo to update the database after disconnecting. (Preferences->Plugins->Customize the Kobo device plugin)”

I am not experienced with Calibre and need a little help with this please. I just need to know which “Kobo device plugin” needs to be customised? I had a look through those Kobo plugins I have installed but cannot see anything that might refer to this new feature.

[davidfor]

You need to configure the KoboTouch driver, or if you use it, the KoboTouchExtended driver. You can do that from the preferences, or, when the device is connected, right-click on the devices button and select the configure option. The option you need to enable is on the "Metadata, on device & advanced" tab. You then need to enable "Update metadata on device" and then "Update metadata on book details" (a terrible name, but I couldn't think of anything better). The options all have tooltips.

[tumbi]

Originally Posted by davidfor

You need to configure the KoboTouch driver, or if you use it, the KoboTouchExtended driver.

Thanks David.

I already had "Update metadata on device" ticked and then I ticked the other box for "Update metadata on book details". But I don't see any difference in the behaviour of Calibre or the Kobo Forma.

When I uploaded some books to the device from Calibre, I was really hoping to have my newly uploaded books showing the series information on the device. But it seems to be behaving as it did before and the Forma displays the appropriate series info only after disconnecting the device and reconnecting.

I guess I thought the "new feature" was going to update the device at the time the new books are added. Or have I missed something?

[John F]

So I wouldn't need to disconnect/reconnect to get series info on a Kobo? Will this work with older firmware, 3.15.0 in particular?

[davidfor]

Sorry, the change notice is incorrect. There is no change to when the metadata is updated. And it is very, very unlikely to ever change.

The change is in how much metadata will be updated. Before, it was only the series info. Now, it will be all the metadata that is visible on the device.

[tumbi]

Oh. That is a pity, but thank you for pointing it out.

I did think it strange to see the update notice (quoted in the first message above) was an announcement by Calibre, and not from a developer of a plugin such as yourself. It looked as if the Calibre program had been updated to include this change as to "when the metadata is updated".

But if you say it is very, very unlikely to change, it does sound like you have addressed this matter before? Maybe one day Kobo will have a look at this in some future firmware update - if it is indeed possible at all?

[davidfor]

Originally Posted by tumbi

Oh. That is a pity, but thank you for pointing it out.

I did think it strange to see the update notice (quoted in the first message above) was an announcement by Calibre, and not from a developer of a plugin such as yourself. It looked as if the Calibre program had been updated to include this change as to "when the metadata is updated".

It is an announcement from calibre as the change is part of calibre. I maintain the KoboTouch driver that is part of calibre. I made the change to that, checked the code in and it was included in the latest update. Unfortunately, it looks like I wasn't clear in the check-in comment and Kovid misinterpreted the change when writing the release notes.

But if you say it is very, very unlikely to change, it does sound like you have addressed this matter before? Maybe one day Kobo will have a look at this in some future firmware update - if it is indeed possible at all?

The reason for not doing it is because, it isn't practical. When the firmware imports the books, it puts the metadata in a database. Until that has been done, there isn't anything for an external application to update. It would be possible to add the details so that the firmware doesn't do the import, but, that is hard, risky, has potential for change and a lot of other things. The only advantage in doing it would be adding the series info when the book is sent. That is a relatively small advantage when you can just plug the device in again and do it.

I have suggested to Kobo a few times that they extend the metadata they read from the book. But, the series metadata is a problem. The epub2 standard does not have any support for series info. Strictly, there isn't anywhere to it from. Calibre does put it in the OPF in application specific attributes, but, whether other developers should read these is a separate issue. I would in an open source application, but, I'm not surprised that a corporation like Kobo doesn't.

Also, this change isn't about what happens when you first send the book to the device. It is more about what happens if you change the metadata later. With this change, if you change the title, comments or something else in the library, the metadata on the device can be updated automatically. My Kobo Utilities plugin could already do this, but, it didn't happen automatically. And I decided to automate it in the driver rather than the plugin.

[anacreon]

@davidfor
I decided to try it, but I am surprised by the results. Since I update the metadata everyday with your plugin for the epubs modified on the day and the day before, I expected only the works modified on the day to be updated. I also tried it with those updated earlier this month, and all were said to have been updated.
Is it because the automatic update modifies different elements from the manual update?
Thanks for the plugin again, anacreon

[compurandom]

So with this update, I now have a virtual book showing up in the device tab.
I can't delete it, and it shows up in the calibre device view both with all checked and all unchecked under metadata / show as on device.

I actually don't care that it shows up; what I care is that it now triggers smart eject to show there's a book on my device that isn't in the library, and I can't seem to figure out how to exclude it. I tried "size:=0" but that doesn't match it.

So, how can I exclude this virtual book in the device view (without excluding real ones), and/or what can I do to help debug where this came from, since it doesn't seem to be a recommendation, archived book, or preview (or at least, unchecking those boxes doesn't make it go away).

[compurandom]

Also, I have a template in update metadata -> comments that the new driver update is not using and clears on the device, so I've disabled the metadata update in the driver for now... if this could be fixed, it would save me a step when adding new books!

[sherman]

Here's a horrifying idea for setting series info on initial book upload.

What if the calibre driver created a SQL trigger (may be an INSTEAD OF, or AFTER INSERT?) for each book it sends, with the appropriate series metadata. That way, when Nickel imports books, the SQL triggers fire and updates the record where appropriate.

I have no idea what the (ongoing) performance impact might be, as I don't know if there is a way of automatically dropping a trigger after it's been successfully fired.

Ugh... I feel filthy just thinking about it. Shower time methinks!

[tumbi]

... or another crazy idea ...

What if the software (Calibre or plugin) performed a sort of “virtual disconnect/reconnect” before the device is Ejected?

It could pretend to disconnect (let the Kobo device think it is disconnected), then perform the reconnect logic (let the Kobo device think it has a new connection), before doing the real Eject.

[sherman]

Originally Posted by tumbi

... or another crazy idea ...

What if the software (Calibre or plugin) performed a sort of “virtual disconnect/reconnect” before the device is Ejected?

It could pretend to disconnect (let the Kobo device think it is disconnected), then perform the reconnect logic (let the Kobo device think it has a new connection), before doing the real Eject.

I don't think this is possible. Calibre really does have to disconnect (unmount) the filesystem before the Kobo can import the content.

It is possible to fake USB plug/unplug events on Kobos, but I don't think it's possible for Calibre to do this. It needs to be done from the device itself, either via SSH/telnet, or a program/script running on the Kobo.

[tumbi]

It looks like we need another patch from Geek1011 to allow it to happen?

[davidfor]

Originally Posted by anacreon

@davidfor
I decided to try it, but I am surprised by the results. Since I update the metadata everyday with your plugin for the epubs modified on the day and the day before, I expected only the works modified on the day to be updated. I also tried it with those updated earlier this month, and all were said to have been updated.
Is it because the automatic update modifies different elements from the manual update?
Thanks for the plugin again, anacreon

The driver will update less metadata than you can in with the plugin. But, the rules are roughly the same. The main difference is some customization and the plugin does other things.

Firstly, the metadata that the driver can update is: title, author, comments/synopsis, series info, subtitle, publisher, published date, ISBN and language.

Then, the driver is only changing common displayed metadata. The plugin is needed for things like the reading direction and the date added. And the driver will always use the plugboard. That matches what happens when a book is sent to the device. Calibre will update the metadata in the book using the plugboard.

There is no template for the comments/synopsis because calibre now supports comments in the metadata plugboard. The subtitle does have a template a there is no other way to do this. Of course, it could be just a column, or a more complex template to have something like the page and word count displayed.

Overall, if you were updating the metadata with the plugboard frequently, you shouldn't expect to many changes when you first turn it on. And after that, there should be even fewer, unless you make a big change somewhere.

One difference I did forget about is that the metadata plugboards can be format specific. So, you can have different plugboards for epubs, kepubs, PDFs and anything else you have on the device. But, because of the way the KoboTouchExtended driver works, if there is no kepub specific plugboard, the epub plugboard will be used.