NickelSeries: Integrated EPUB/KEPUB series metadata support

[geek1011]

NickelSeries
.
I've been working on a small mod to add built-in support for parsing series (and subtitle) metadata on sideloaded books.

To install it, copy KoboRoot.tgz to KOBOeReader/.kobo the same way you would with a firmware update.
.
Features:

• Works on EPUB and KEPUB files.
• Compatible with the Calibre Kobo plugin.
• Compatible with Calibre or EPUB3 series metadata (Calibre takes precedence).
• Will not override Nickel if Kobo ends up implementing it.
• Compatible with firmware 3.19.5761+.
• Survives firmware updates.
• Does not modify any system files, single file only.
• Easy to uninstall (either delete it over telnet/ssh or create a file named `ns_uninstall` in the KOBOeReader partition).
• Metadata appears instantly.
• Will also add subtitle metadata (EPUB3 or a Calibre custom column named `#subtitle`).

Troubleshooting:

• How do I uninstall NickelSeries?
  Create a file named `ns_uninstall` (without any extension) in the KOBOeReader partition and reboot.
• How do I uninstall NickelSeries manually?
  Connect over Telnet/SSH and delete `/usr/local/Kobo/imageformats/libns.so`. A
  hard factory reset will also remove it.
• Series metadata is incorrect or missing for imported books.
  Please report the issue and post a copy of the OPF document from within the EPUB.
• I installed NickelSeries and nothing changed!
  NickelSeries doesn't parse metadata for existing books. To do this, you can use seriesmeta with the argument `--no-persist`.
• The series metadata changes after a reboot or USB connection.
  If you use Calibre or seriesmeta, those will take precedence.
• Will this conflict with Calibre or Kobo-UNCaGED?
  No. For the initial import, NickelSeries prioritizes the `calibre:series` meta tags. For changes afterwards, NickelSeries doesn't override them.
• My Kobo crashes when I try and import a book.
  This issue is unlikely to occur, but if it does, you can uninstall NickelSeries the usual way (see above). If possible, it would also be helpful if you could report the error and post the syslog.

Changelog:

• v1 (2020-07-19, 8c5d318): First stable version, all features implemented.
• v2 (2020-07-24, 3c77ac0): Switched to NickelHook, improved metadata parsing, added support for subtitle metadata, fixed parsing books with Unicode filenames.
• v3 (2020-08-12, a06f1ac): Added support for using series metadata without an index.
• v4 (2023-10-13, 0deffca): Fix bug in EPUB3 series metadata parsing due to undefined behavior.

Download | Source Code
.

[DevonHess]

As a Calibre user, I didn't even know this was a problem. Series support is the best update we've gotten since I started using Kobo, so great work on this.

[NiLuJe]

@DevonHess: Long-time Calibre+Kobo users are probably used to do their USBMS sessions in pairs to let the Calibre driver fixup metadata on the second pass .

[JSWolf]

Originally Posted by NiLuJe

@DevonHess: Long-time Calibre+Kobo users are probably used to do their USBMS sessions in pairs to let the Calibre driver fixup metadata on the second pass .

NickelMenu is really great for series on Kobo as it allows you to reconnect your Kobo to Calibre for the series update without having to unplug and replug the USB cable. It's done in software. Works very well.

[geek1011]

Originally Posted by JSWolf

NickelMenu is really great for series on Kobo as it allows you to reconnect your Kobo to Calibre for the series update without having to unplug and replug the USB cable. It's done in software. Works very well.

Eventually, I got tired of doing that too, especially with the slow SD card (seriesmeta is painfully slow on-device) and the USB connection issues I've been having. That's the main reason why I made this. NickelSeries makes the process work as well as if Kobo had implemented it themselves (possible better, as NS supports Calibre's metadata as well as the less-common official EPUB3 one).

I had the idea for this last year, but I didn't want it enough for it to be worth implementing the hooks, building the toolchain, and so on. Once we made NickelMenu, most of the groundwork (toolchain, Qt sysroot, runtime hooking, library loading, failsafe) was already done, leaving just the actual hook and series metadata parsing. A few months ago, I did the KEPUB part in an hour or two, but put it aside in favour of working on more NM features (Adobe EPUB is significantly more complicated to do). Yesterday, I finished implementing Adobe EPUB support and cleaning up the code.

[MGlitch]

Ok so explain this to me like I’m five kind of moment. I’m gathering this does what Calibre does with proper set up on a second connection after loading books.

In which case I’d suspect Sherman would be interested in implementing it in his KoboUnCaged project. Which currently requires a second wireless connection to sync metadata from Calibre.

Or am I missing what this does entirely/mostly/somewhat?

[geek1011]

Originally Posted by MGlitch

Ok so explain this to me like I’m five kind of moment. I’m gathering this does what Calibre does with proper set up on a second connection after loading books.

In which case I’d suspect Sherman would be interested in implementing it in his KoboUnCaged project. Which currently requires a second wireless connection to sync metadata from Calibre.

Or am I missing what this does entirely/mostly/somewhat?

Yes, kind of. This doesn't edit the database directly. Instead, it literally makes series metadata support built-in to the import process by intercepting Nickel's built-in metadata parser.

Edit: To clarify, it means that this will work with UNCaGED without any changes (Sherman's already tested it).

[mbovenka]

Originally Posted by geek1011

Yes, kind of. This doesn't edit the database directly. Instead, it literally makes series metadata support built-in to the import process by intercepting Nickel's built-in metadata parser.

Would it be possible to support 'subtitle' as well? AFAIK, that takes a second connect too.

[geek1011]

Originally Posted by mbovenka

Would it be possible to support 'subtitle' as well? AFAIK, that takes a second connect too.

Yes, that would be possible. I'll look into the details later. Are you referring to the EPUB3 subtitle metadata (refines="#...", title-type=subtitle) only? IIRC, Calibre doesn't natively support subtitles.

For the record, anything involving metadata from the EPUB OPF document which Kobo doesn't support is within scope for NickelSeries.

[jackie_w]

Originally Posted by geek1011

Yes, that would be possible. I'll look into the details later. Are you referring to the EPUB3 subtitle metadata (refines="#...", title-type=subtitle) only? IIRC, Calibre doesn't natively support subtitles.

For the record, anything involving metadata from the EPUB OPF document which Kobo doesn't support is within scope for NickelSeries.

@mbovenka will need to clarify this but I think he may be talking about the Subtitle feature in the calibre driver and/or the Kobo Utilities plugin. If this is true then that data never appears in the epub/kepub OPF file. It's a calibre template which @davidfor's code writes directly to the Kobo database, but only on 2nd, 3rd etc USB connections.

[xyclonei]

Thank you very much for this! This mod is such a timesaver as compared to seriesmeta.

[mbovenka]

Originally Posted by jackie_w

@mbovenka will need to clarify this but I think he may be talking about the Subtitle feature in the calibre driver and/or the Kobo Utilities plugin. If this is true then that data never appears in the epub/kepub OPF file. It's a calibre template which @davidfor's code writes directly to the Kobo database, but only on 2nd, 3rd etc USB connections.

Yes, that's what I am talking about.

[mbovenka]

Originally Posted by geek1011

For the record, anything involving metadata from the EPUB OPF document which Kobo doesn't support is within scope for NickelSeries.

As I understand now, the 'subtitle' info I meant (the info the Kobo Utilities plugin & Kobo Extended driver can put in) never hits the OPF file and is a direct DB update.

So I assume it's off the table? Pity (Unless @davidfor could be persuaded to put that info in the OPF file so NickelSeries can get at it...perhaps you two should have a chat? I wish I could do Python!)

[davidfor]

Originally Posted by mbovenka

As I understand now, the 'subtitle' info I meant (the info the Kobo Utilities plugin & Kobo Extended driver can put in) never hits the OPF file and is a direct DB update.

So I assume it's off the table? Pity (Unless @davidfor could be persuaded to put that info in the OPF file so NickelSeries can get at it...perhaps you two should have a chat? I wish I could do Python!)

Setting the subtitle from the driver uses a template. If the template is just a custom column, the column will be in the OPF. It shouldn't be to hard to get the value of the column from the OPF. Doing that would be better than the driver adding the value somewhere.

[mbovenka]

Originally Posted by davidfor

Setting the subtitle from the driver uses a template. If the template is just a custom column, the column will be in the OPF. It shouldn't be to hard to get the value of the column from the OPF. Doing that would be better than the driver adding the value somewhere.

I could live with that I'm not using a custom column now (just a template putting everything from the title after the first colon in the subtitle), but I could easily do that.

@geek1011, please pretty please?

(Edit: now putting the same info in a custom column called (surprise) 'subtitle' and using the template '{#subtitle}' in the driver config.)