Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Readers > Kobo Reader

Notices

Reply
 
Thread Tools Search this Thread
Old 09-22-2018, 11:46 AM   #1
nikev
Connoisseur
nikev began at the beginning.
 
Posts: 53
Karma: 10
Join Date: Sep 2009
Device: PRS 505
Kobo Utilities Plugin Question

Hello,

I just installed the Kobo Utilities plugin, with the hope of using it to update metadata on my device (Aura H2O Edition 1).

However, I can't figure out how to enable this function. I understand that I have to do this from the device list, but I don't know exactly what that means.

I am looking at the books on the device as shown by Calibre's "Show books in main memory of device" option, and have selected a book, but the update metadata option in the Utilities plugin is greyed out.

What am I doing wrong?

Thanks in advance,

Nike
nikev is offline   Reply With Quote
Old 09-22-2018, 02:41 PM   #2
DNSB
Bibliophagist
DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.DNSB ought to be getting tired of karma fortunes by now.
 
DNSB's Avatar
 
Posts: 37,549
Karma: 149859178
Join Date: Jul 2010
Location: Vancouver
Device: Kobo Sage, Libra Colour, Lenovo M8 FHD, Paperwhite 4, Tolino epos
Quote:
Originally Posted by nikev View Post
Hello,

I just installed the Kobo Utilities plugin, with the hope of using it to update metadata on my device (Aura H2O Edition 1).

However, I can't figure out how to enable this function. I understand that I have to do this from the device list, but I don't know exactly what that means.

I am looking at the books on the device as shown by Calibre's "Show books in main memory of device" option, and have selected a book, but the update metadata option in the Utilities plugin is greyed out.

What am I doing wrong?

Thanks in advance,

Nike
Select the books from the calibre library view not from the books on device view.
DNSB is offline   Reply With Quote
Advert
Old 09-22-2018, 06:12 PM   #3
nikev
Connoisseur
nikev began at the beginning.
 
Posts: 53
Karma: 10
Join Date: Sep 2009
Device: PRS 505
Thanks for the clarification; I must have misunderstood the instructions.

However, I am still confused about what the plugin does and how it interacts with the reader.

What I am trying to do is fix a bunch (quite a lot) on my reader where the author's first name and last name are mixed up. I am not sure how this happened in the first place; looking at the Calibre files, probably it's because some used the format for author as

last name, first name

while others use

first name last name

Also there was no consistency in Calibre files in the format for author sort.

I have now fixed the metadata in Calibre so all use the author field in the format "first name last name" while the author sort key is in the format "last name, first name"

I'd now like to push these over to the device, so sorted author lists make sense.

I used the "update metadata on device" function, and it seems to have worked on some books but not on others.

Is there some other way to fix this, short of wiping my library, or removing the problem books, and side loading them again?

Thanks,

Nike
nikev is offline   Reply With Quote
Old 09-22-2018, 09:33 PM   #4
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,906
Karma: 47303822
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by nikev View Post
However, I am still confused about what the plugin does and how it interacts with the reader.
When you open the menu for the plugin, the functions available depend on the list you are looking at. Most act on the books on the device. But, those that need books in the calibre library are only available when looking at the library list. Those that only act on books on the device and don't need the book to be in calibre are only available on the device list. There is some overlap and some functions that are on both, but might behave a little differently.

The metadata update only works for books that are in the calibre library and on the device, hence it is only available when looking at the library.
Quote:
What I am trying to do is fix a bunch (quite a lot) on my reader where the author's first name and last name are mixed up. I am not sure how this happened in the first place; looking at the Calibre files, probably it's because some used the format for author as

last name, first name

while others use

first name last name

Also there was no consistency in Calibre files in the format for author sort.

I have now fixed the metadata in Calibre so all use the author field in the format "first name last name" while the author sort key is in the format "last name, first name"

I'd now like to push these over to the device, so sorted author lists make sense.

I used the "update metadata on device" function, and it seems to have worked on some books but not on others.

Is there some other way to fix this, short of wiping my library, or removing the problem books, and side loading them again?
The update is updating the database on the device. When doing this, it compares the values in the library and on the device and only makes a change if they are different. It should work, but it won't surprise me much if there is a case where it fails. I would need to see some examples where it failed to have an idea of why. The best way to do this is to run calibre in debug mode, run the function on the ones that failed and post the log for me to look at.

Unless the sorting problem is with books that have multiple authors. In this case, the default way that calibre populates the author doesn't work well. The device expects the authors as "FN1 LN1, FN2 LN2". Calibre is more likely to use "FN1 LN1 & FN2 LN2". Because of this, I recommend using the following metadata plugboard to set the author:

Code:
program: raw_list('authors', ', ')
davidfor is offline   Reply With Quote
Old 09-22-2018, 11:26 PM   #5
nikev
Connoisseur
nikev began at the beginning.
 
Posts: 53
Karma: 10
Join Date: Sep 2009
Device: PRS 505
Thanks. I'm happy to run in debug mode, if you can tell me exactly what tests you would like me to do.

In case it's helpful, here are some examples of what's happening. My impression is that Kobo is doing something odd in the way it uses metadata to sort lists.

For instance, if I go to My Books, select the Author's tab, and sort by Last Name, the first entry I see is Bruno A.

This is wrong; the author's name is Bruno A. Cayoun. When I select the book, his name is displayed as Cayoun, Bruno A. This book is an Kepub purchased from Kobo.

Similarly, I get a bunch of books by an author whose last name is shown in the list as "Ann." This is again wrong; that's the author's first name, and clicking on the name displays the author's name as Cleeves, Ann.

In Calibre the author's name is Ann Cleeves, and the author sort key is Cleeves, Ann. Trying to manually update the metadata doesn't do anything.

The plugin report is this:

calibre, version 3.31.0
Kobo Utilities - Device library updated: Update summary:
Books updated=0
Unchanged books=4
Books not on device=0
Total books=4

I had the same thing happen with Gutenberg books by Jane Austen. I fixed those in Calibre, deleted the books from the reader, and reloaded them. That solved the problem, but it's a pain to do for everything.

However (and this may be a hint as to where the problem lies) the My Books > Author > Last name yields "Jane Austen" and clicking on the books gives the author's name as "Jane Austen." This is different from the problem behaviour, where the author's name is simply listed in the sorted list as (say) Ann and the shown upon clicking the book is "Cleeves, Ann"

Any ideas for a simple fix to this would be welcome.

Nike
nikev is offline   Reply With Quote
Advert
Old 09-22-2018, 11:30 PM   #6
nikev
Connoisseur
nikev began at the beginning.
 
Posts: 53
Karma: 10
Join Date: Sep 2009
Device: PRS 505
David:

I have the debug output from the attempt to fix metadata that I described. It's a bit long, and perhaps ought not to be pasted in to the forum; if there's some other way to get it to you, I'd be happy to do so.

Nike
nikev is offline   Reply With Quote
Old 09-22-2018, 11:48 PM   #7
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,906
Karma: 47303822
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by nikev View Post
I have the debug output from the attempt to fix metadata that I described. It's a bit long, and perhaps ought not to be pasted in to the forum; if there's some other way to get it to you, I'd be happy to do so.
The best way is to upload it somewhere and send me a PM with a link. Otherwise PM me for an email address to send it to.
davidfor is offline   Reply With Quote
Old 09-24-2018, 01:55 AM   #8
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,906
Karma: 47303822
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
I have had a look at the logs you sent, and it appears to be working correctly. The problem is the options you have and the data in the library.

Firstly, the options:

Code:
DEBUG:   33.7 _update_metadata: number books= 4 options= {u'rating': False, u'title': True, u'series': True, u'descriptionUseTemplate': False, u'reading_direction': u'Default', u'set_sync_date': False, u'sync_date_library_date': u'timestamp', u'set_reading_direction': False, u'usePlugboard': False, u'isbn': False, u'description': False, u'update_KoboEpubs': True, u'published_date': False, u'descriptionTemplate': u'', u'subtitleTemplate': u'', u'subtitle': False, u'mark_not_interested': False, u'titleSort': True, u'publisher': False, u'authourSort': True, u'language': False, u'resetPosition': False, u'readingStatus': -1, u'author': True, u'setRreadingStatus': False}
The bits marked in lime shows that you are updating the author and setting the metadata to the author sort field. The other marked option shows that you are updating books purchased and downloaded to the device as well as sideloaded book.

The following is a section of the log showing what happens for one of the books by Ann Cleeves.

Code:
DEBUG:   33.8 generate_metadata_query - test_query=SELECT Title,       Attribution,     Description,     Publisher,    Series,        SeriesNumber,  Subtitle,  ReadStatus,  DateCreated,  Language,  PageProgressDirection,  ___SyncTime,  ISBN,  FeedbackType,  FeedbackTypeSynced,  r.Rating,  r.DateModified FROM content c1  left outer join ratings r on c1.ContentID = r.ContentID WHERE c1.BookId IS NULL AND c1.ContentID = ?
DEBUG:   33.8 _update_metadata: searching for contentId='file:///mnt/onboard/Cleeves, Ann/Bird in the Hand, A - Ann Cleeves.kepub.epub'
DEBUG:   33.8 _update_metadata: self.options[cfg.KEY_UDPATE_KOBO_EPUBS]= True
DEBUG:   33.8 _update_metadata: contentID.startswith('file')= True
DEBUG:   33.8 _update_metadata: found contentId='file:///mnt/onboard/Cleeves, Ann/Bird in the Hand, A - Ann Cleeves.kepub.epub'
DEBUG:   33.8     result= {u'Series': None, u'Attribution': u'Cleeves, Ann', u'Subtitle': None, u'ReadStatus': 1, u'Language': u'en', u'Title': u'Bird in the Hand, A', u'___SyncTime': u'2018-08-14T23:27:50Z', u'Description': None, u'PageProgressDirection': None, u'ISBN': None, u'Publisher': None, u'DateCreated': None, u'SeriesNumber': None, u'DateModified': None, u'FeedbackTypeSynced': 0, u'Rating': None, u'FeedbackType': 0}
DEBUG:   33.8     result['Title']='Bird in the Hand, A'
DEBUG:   33.8     result['Attribution']='Cleeves, Ann'
DEBUG:   33.8 _update_metadata: author= [u'Ann Cleeves']
DEBUG:   33.8 _update_metadata: using author_sort= Cleeves, Ann
DEBUG:   33.8 _update_metadata: using author_sort - author= [u'Ann Cleeves']
DEBUG:   33.8 _update_metadata: title_string= Bird in the Hand, A
DEBUG:   33.8 _update_metadata: title_string= Bird in the Hand, A
DEBUG:   33.8 _update_metadata: authors_string= Cleeves, Ann
DEBUG:   33.8 _update_metadata: authors_string= Cleeves, Ann
Marked in lime is the author name on the device - "Cleeves, Ann". In the calibre library, you have "Ann Cleeves", but the author sort is "Cleeves, Ann". As your options are to use the author sort, that is what is used when updating the metadata on the device. And as the library value matches the device value, no change is made.

Based on your earlier post, you want to use "FN LN" on the device. Which is the better choice for recent firmware. If that is the case, then don't select the "Author sort" option on the metadata update dialog.
davidfor is offline   Reply With Quote
Old 09-25-2018, 11:55 AM   #9
nikev
Connoisseur
nikev began at the beginning.
 
Posts: 53
Karma: 10
Join Date: Sep 2009
Device: PRS 505
Thanks for taking the time to look at this.

Just to be clear: Calibre seems to distinguish between author name and author sort explicitly. What I am understanding from your post above is that Kobo internally uses the author name alone, and calculated the sort field from that. Am I correct in this? And if I check the "use author sort" option in the plugin, it pushes that over to the Kobo rather than the author name?

Nike
nikev is offline   Reply With Quote
Old 09-25-2018, 07:52 PM   #10
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,906
Karma: 47303822
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by nikev View Post
Just to be clear: Calibre seems to distinguish between author name and author sort explicitly. What I am understanding from your post above is that Kobo internally uses the author name alone, and calculated the sort field from that. Am I correct in this? And if I check the "use author sort" option in the plugin, it pushes that over to the Kobo rather than the author name?
That is correct. Overall:
  • Calibre uses one dc:contributor tag for each author.
  • Calibre puts the author sort in the "file-as" attribute in which of the first dc:contributor tag.
  • The device reads the author from the first dc:contributor tag in the metadata and stores it in the database.
  • The device only uses the value of this tag and ignores the other attributes.
  • It calculates the sorting from this author and assumes it is "FN LN" or "FN1 LN1, FN2 LN2".
  • If the author is "LN, FN" this will be treated as two authors "LN" and "FN". Books will sort by author correctly, but, the author list will show both authors.
  • The metadata update in the Kobo Utilities plugin directly updates the database. Depending on the option chosen, whatever is in the author, author sort or calculated by the metadata plugboard is used.

Because of this, I recommend using the metadata plugboard I posted earlier in the thread. It seems to give the best results on the device.
davidfor is offline   Reply With Quote
Old 09-25-2018, 11:55 PM   #11
nikev
Connoisseur
nikev began at the beginning.
 
Posts: 53
Karma: 10
Join Date: Sep 2009
Device: PRS 505
Thanks very much for your time and help. It all works well now. Somehow I got the impression that Kobo had the same two fields author and author sort as Calibre, which is where the confusion about the function of the plugin came.

Nike
nikev is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Kobo Utilities davidfor Plugins 3013 Today 01:31 PM
Kobo Hacks and Utilities Index ItalianUruguayan Kobo Reader 26 08-09-2020 12:38 PM
Aura H2O Kobo Utilities reporting 2 books not on device Boneist Kobo Reader 11 10-29-2017 06:33 PM
Calibre Kobo Utilities - Wishlist!! Stevex Kobo Reader 42 10-17-2017 06:57 AM
Help with Kobo Utilities needed chiara_sc Kobo Reader 3 09-02-2016 07:05 PM


All times are GMT -4. The time now is 02:28 PM.


MobileRead.com is a privately owned, operated and funded community.