Kobo Utilities Plugin Question

[nikev]

Hello,

I just installed the Kobo Utilities plugin, with the hope of using it to update metadata on my device (Aura H2O Edition 1).

However, I can't figure out how to enable this function. I understand that I have to do this from the device list, but I don't know exactly what that means.

I am looking at the books on the device as shown by Calibre's "Show books in main memory of device" option, and have selected a book, but the update metadata option in the Utilities plugin is greyed out.

What am I doing wrong?

Thanks in advance,

Nike

[DNSB]

Originally Posted by nikev

Hello,

I just installed the Kobo Utilities plugin, with the hope of using it to update metadata on my device (Aura H2O Edition 1).

However, I can't figure out how to enable this function. I understand that I have to do this from the device list, but I don't know exactly what that means.

I am looking at the books on the device as shown by Calibre's "Show books in main memory of device" option, and have selected a book, but the update metadata option in the Utilities plugin is greyed out.

What am I doing wrong?

Thanks in advance,

Nike

Select the books from the calibre library view not from the books on device view.

[nikev]

Thanks for the clarification; I must have misunderstood the instructions.

However, I am still confused about what the plugin does and how it interacts with the reader.

What I am trying to do is fix a bunch (quite a lot) on my reader where the author's first name and last name are mixed up. I am not sure how this happened in the first place; looking at the Calibre files, probably it's because some used the format for author as

last name, first name

while others use

first name last name

Also there was no consistency in Calibre files in the format for author sort.

I have now fixed the metadata in Calibre so all use the author field in the format "first name last name" while the author sort key is in the format "last name, first name"

I'd now like to push these over to the device, so sorted author lists make sense.

I used the "update metadata on device" function, and it seems to have worked on some books but not on others.

Is there some other way to fix this, short of wiping my library, or removing the problem books, and side loading them again?

Thanks,

Nike

[davidfor]

Originally Posted by nikev

However, I am still confused about what the plugin does and how it interacts with the reader.

When you open the menu for the plugin, the functions available depend on the list you are looking at. Most act on the books on the device. But, those that need books in the calibre library are only available when looking at the library list. Those that only act on books on the device and don't need the book to be in calibre are only available on the device list. There is some overlap and some functions that are on both, but might behave a little differently.

The metadata update only works for books that are in the calibre library and on the device, hence it is only available when looking at the library.

What I am trying to do is fix a bunch (quite a lot) on my reader where the author's first name and last name are mixed up. I am not sure how this happened in the first place; looking at the Calibre files, probably it's because some used the format for author as

last name, first name

while others use

first name last name

Also there was no consistency in Calibre files in the format for author sort.

I have now fixed the metadata in Calibre so all use the author field in the format "first name last name" while the author sort key is in the format "last name, first name"

I'd now like to push these over to the device, so sorted author lists make sense.

I used the "update metadata on device" function, and it seems to have worked on some books but not on others.

Is there some other way to fix this, short of wiping my library, or removing the problem books, and side loading them again?

The update is updating the database on the device. When doing this, it compares the values in the library and on the device and only makes a change if they are different. It should work, but it won't surprise me much if there is a case where it fails. I would need to see some examples where it failed to have an idea of why. The best way to do this is to run calibre in debug mode, run the function on the ones that failed and post the log for me to look at.

Unless the sorting problem is with books that have multiple authors. In this case, the default way that calibre populates the author doesn't work well. The device expects the authors as "FN1 LN1, FN2 LN2". Calibre is more likely to use "FN1 LN1 & FN2 LN2". Because of this, I recommend using the following metadata plugboard to set the author:

program: raw_list('authors', ', ')

[nikev]

Thanks. I'm happy to run in debug mode, if you can tell me exactly what tests you would like me to do.

In case it's helpful, here are some examples of what's happening. My impression is that Kobo is doing something odd in the way it uses metadata to sort lists.

For instance, if I go to My Books, select the Author's tab, and sort by Last Name, the first entry I see is Bruno A.

This is wrong; the author's name is Bruno A. Cayoun. When I select the book, his name is displayed as Cayoun, Bruno A. This book is an Kepub purchased from Kobo.

Similarly, I get a bunch of books by an author whose last name is shown in the list as "Ann." This is again wrong; that's the author's first name, and clicking on the name displays the author's name as Cleeves, Ann.

In Calibre the author's name is Ann Cleeves, and the author sort key is Cleeves, Ann. Trying to manually update the metadata doesn't do anything.

The plugin report is this:

calibre, version 3.31.0
Kobo Utilities - Device library updated: Update summary:
Books updated=0
Unchanged books=4
Books not on device=0
Total books=4

I had the same thing happen with Gutenberg books by Jane Austen. I fixed those in Calibre, deleted the books from the reader, and reloaded them. That solved the problem, but it's a pain to do for everything.

However (and this may be a hint as to where the problem lies) the My Books > Author > Last name yields "Jane Austen" and clicking on the books gives the author's name as "Jane Austen." This is different from the problem behaviour, where the author's name is simply listed in the sorted list as (say) Ann and the shown upon clicking the book is "Cleeves, Ann"

Any ideas for a simple fix to this would be welcome.

Nike

[nikev]

David:

I have the debug output from the attempt to fix metadata that I described. It's a bit long, and perhaps ought not to be pasted in to the forum; if there's some other way to get it to you, I'd be happy to do so.

Nike

[davidfor]

Originally Posted by nikev

I have the debug output from the attempt to fix metadata that I described. It's a bit long, and perhaps ought not to be pasted in to the forum; if there's some other way to get it to you, I'd be happy to do so.

The best way is to upload it somewhere and send me a PM with a link. Otherwise PM me for an email address to send it to.

[davidfor]

I have had a look at the logs you sent, and it appears to be working correctly. The problem is the options you have and the data in the library.

Firstly, the options:

DEBUG:   33.7 _update_metadata: number books= 4 options= {u'rating': False, u'title': True, u'series': True, u'descriptionUseTemplate': False, u'reading_direction': u'Default', u'set_sync_date': False, u'sync_date_library_date': u'timestamp', u'set_reading_direction': False, u'usePlugboard': False, u'isbn': False, u'description': False, u'update_KoboEpubs': True, u'published_date': False, u'descriptionTemplate': u'', u'subtitleTemplate': u'', u'subtitle': False, u'mark_not_interested': False, u'titleSort': True, u'publisher': False, u'authourSort': True, u'language': False, u'resetPosition': False, u'readingStatus': -1, u'author': True, u'setRreadingStatus': False}

The bits marked in lime shows that you are updating the author and setting the metadata to the author sort field. The other marked option shows that you are updating books purchased and downloaded to the device as well as sideloaded book.

The following is a section of the log showing what happens for one of the books by Ann Cleeves.

DEBUG:   33.8 generate_metadata_query - test_query=SELECT Title,       Attribution,     Description,     Publisher,    Series,        SeriesNumber,  Subtitle,  ReadStatus,  DateCreated,  Language,  PageProgressDirection,  ___SyncTime,  ISBN,  FeedbackType,  FeedbackTypeSynced,  r.Rating,  r.DateModified FROM content c1  left outer join ratings r on c1.ContentID = r.ContentID WHERE c1.BookId IS NULL AND c1.ContentID = ?
DEBUG:   33.8 _update_metadata: searching for contentId='file:///mnt/onboard/Cleeves, Ann/Bird in the Hand, A - Ann Cleeves.kepub.epub'
DEBUG:   33.8 _update_metadata: self.options[cfg.KEY_UDPATE_KOBO_EPUBS]= True
DEBUG:   33.8 _update_metadata: contentID.startswith('file')= True
DEBUG:   33.8 _update_metadata: found contentId='file:///mnt/onboard/Cleeves, Ann/Bird in the Hand, A - Ann Cleeves.kepub.epub'
DEBUG:   33.8     result= {u'Series': None, u'Attribution': u'Cleeves, Ann', u'Subtitle': None, u'ReadStatus': 1, u'Language': u'en', u'Title': u'Bird in the Hand, A', u'___SyncTime': u'2018-08-14T23:27:50Z', u'Description': None, u'PageProgressDirection': None, u'ISBN': None, u'Publisher': None, u'DateCreated': None, u'SeriesNumber': None, u'DateModified': None, u'FeedbackTypeSynced': 0, u'Rating': None, u'FeedbackType': 0}
DEBUG:   33.8     result['Title']='Bird in the Hand, A'
DEBUG:   33.8     result['Attribution']='Cleeves, Ann'
DEBUG:   33.8 _update_metadata: author= [u'Ann Cleeves']
DEBUG:   33.8 _update_metadata: using author_sort= Cleeves, Ann
DEBUG:   33.8 _update_metadata: using author_sort - author= [u'Ann Cleeves']
DEBUG:   33.8 _update_metadata: title_string= Bird in the Hand, A
DEBUG:   33.8 _update_metadata: title_string= Bird in the Hand, A
DEBUG:   33.8 _update_metadata: authors_string= Cleeves, Ann
DEBUG:   33.8 _update_metadata: authors_string= Cleeves, Ann

Marked in lime is the author name on the device - "Cleeves, Ann". In the calibre library, you have "Ann Cleeves", but the author sort is "Cleeves, Ann". As your options are to use the author sort, that is what is used when updating the metadata on the device. And as the library value matches the device value, no change is made.

Based on your earlier post, you want to use "FN LN" on the device. Which is the better choice for recent firmware. If that is the case, then don't select the "Author sort" option on the metadata update dialog.

[nikev]

Thanks for taking the time to look at this.

Just to be clear: Calibre seems to distinguish between author name and author sort explicitly. What I am understanding from your post above is that Kobo internally uses the author name alone, and calculated the sort field from that. Am I correct in this? And if I check the "use author sort" option in the plugin, it pushes that over to the Kobo rather than the author name?

Nike

[davidfor]

Originally Posted by nikev

Just to be clear: Calibre seems to distinguish between author name and author sort explicitly. What I am understanding from your post above is that Kobo internally uses the author name alone, and calculated the sort field from that. Am I correct in this? And if I check the "use author sort" option in the plugin, it pushes that over to the Kobo rather than the author name?

That is correct. Overall:

• Calibre uses one dc:contributor tag for each author.
• Calibre puts the author sort in the "file-as" attribute in which of the first dc:contributor tag.
• The device reads the author from the first dc:contributor tag in the metadata and stores it in the database.
• The device only uses the value of this tag and ignores the other attributes.
• It calculates the sorting from this author and assumes it is "FN LN" or "FN1 LN1, FN2 LN2".
• If the author is "LN, FN" this will be treated as two authors "LN" and "FN". Books will sort by author correctly, but, the author list will show both authors.
• The metadata update in the Kobo Utilities plugin directly updates the database. Depending on the option chosen, whatever is in the author, author sort or calculated by the metadata plugboard is used.

Because of this, I recommend using the metadata plugboard I posted earlier in the thread. It seems to give the best results on the device.

[nikev]

Thanks very much for your time and help. It all works well now. Somehow I got the impression that Kobo had the same two fields author and author sort as Calibre, which is where the confusion about the function of the plugin came.

Nike