How can I more reliably show bks "on device"? (Kobo Forma)

[LivresInOz]

Often the books just don't show up with the green tick in the 'On Device' column, even tho they are in fact on the Kobo Forma.

Is there a way to fix?
Its a problem cos I have a lot of books and it requires me to check the device or just re-sideload JIC.

Apologies if asked before.

[theducks]

Only books that are sideloaded are guaranteed (direct downloads may be virtual. protected storage)
In device view: right click one without a tic: match to Library:select the match
repeat / spin until done

[LivresInOz]

Originally Posted by theducks

Only books that are sideloaded are guaranteed (direct downloads may be virtual. protected storage)
In device view: right click one without a tic: match to Library:select the match
repeat / spin until done

Thanks I know about this but its a nightmare cos I have so many books.

[davidfor]

Calibre automatically tries to match the books when the device is connected. It does this using the title and the author. If those do not match, then the book doesn't match. If you sideload the books, then they should match. But, if you source the books elsewhere, such as from Kobo, then it depends on the metadata they supply. And that usually means the title has something extra in it.

There is one thing specific to Kobo devices, and that is how it handles multiple authors. When importing sideloaded books, Kobo only reads the first dc:creator tag. As calibre writes each author into separate dc:creator tags, that means the device will only see one author. And that means matching the book will probably fail. To get around this, I recommend using a metadata plugboard to set the author who Kobo will read it. The template to use in the metadata plugboard is:

program: raw_list('authors', ', ')

Use that to set the author.

I have found that doing this reduces the number of mismatches to a manageable level.

[LivresInOz]

Thanks David - actually I did vaguely remember what you said before about multiple authors and I did get some success just from that info.

I noticed especially that the books that were having problem often had "&" or "and" in the author metadata. When I got rid of that (and re-sideloaded), it seemed to find the books. But maybe I was just doing that by deleting all authors but one, which created the same effect you speak of?

Frankly, if its easier, I may just do that.

HOWEVER, it seems that your other suggestion is better long-term. The only thing is that plugboards are a total mystery to me.

THIS BIT MAY NOT BE NECESSARY - PLS SEE NEXT POST:
Would it be possible to just chuck me some short hand that will give me a clue how to do this? Eg.

Menu blah > menu blah > button blah >

Then I can probably work out how to do it.

[LivresInOz]

Actually I found this help page and it doesnt look so hard.

Only questions are:
- Device: for my FORMA do i choose Kobo or Kobotouchextended? [Q1]
- Destination: Authors? Or Author_sort? [Q2]


Q3. Do I have to do anything else?


ADDITIONAL Q if you see fit:

Q4. Does anything appear in the main interface to show me that ive fixed the problem as you suggested - for example the authors appear differently in the column?

Q5. What is this actually doing?

[ownedbycats]

If you use the KTE driver, set your plugboards to KoboTouchExtended.

[LivresInOz]

Originally Posted by ownedbycats

If you use the KTE driver, set your plugboards to KoboTouchExtended.

Thanks - but no idea what that is.
I have:
- a Kobo Forma
- running latest firmware
- Koreader and Plato
- some custom nickelmenu items i made

I still need to know whether to choose:
Kobo or Kobotouchextended
Author or Author_sort

[davidfor]

Originally Posted by LivresInOz

Actually I found this help page and it doesnt look so hard.

Only questions are:
- Device: for my FORMA do i choose Kobo or Kobotouchextended? [Q1]
- Destination: Authors? Or Author_sort? [Q2]

For the driver, you choose whichever you are using. Both work with all Kobo eredears starting with the Touch. The difference is that the KoboTouchExtended driver transforms epubs to kepubs when they are being sent to the device. Check which of the drivers you have configured to use then setup the plugboard for it. Or, you could create the plugboard for "any device" and it will work with both, but will also with other devices.

If you don't know, then when the device is connected, right-click the device button and choose "Configure this device". The active driver configuration dialog will be displayed. If it has a tab called "Extended", then it is the KoboTouchExtended .

As I said above, put it in "authors". The Kobo devices do not look at where the author_sort is stored.

Q3. Do I have to do anything else?


ADDITIONAL Q if you see fit:

Q4. Does anything appear in the main interface to show me that ive fixed the problem as you suggested - for example the authors appear differently in the column?

Q5. What is this actually doing?

When a book is sent outside of calibre (by send-to-device or save-to-disk or other methods) the metadata in the book is updated. The metadata plugboard is used to make changes to that metadata. It does not affect what is in the calibre library, only the copy of the book created outside of calibre. You would see the changes by editing the copy on the device. Or when the device imports it and shows the details in the UI. On the device, you will see the multiple authors as "FN1 LN1, FN2 LN2".

When you connect the device, the matching should work better and more books will show with the green tick indicating they are matched. If you go to the device list, the same will be the case. Here, when you look, multiple authors will show as "FN1 LN1 & FN2 LN2" because that is how calibre shows them.

There will still be unmatched books. That usually comes down to a difference in the title or author. For example, I have a copy of Jim Butcher's "Small Favor". That didn't match. It was because I had bought the book from Kobo's Australian store and the publisher uses the UK spelling of "Favour" but, when I added the book to calibre, the metadata downloaded used the US spelling. Hence the book didn't automatically match. For simplicity, I changed the spelling in calibre so the match would happen.

[LivresInOz]

What worked for me and didn't, in trying to make sure books were detected as present on the Kobo (Forma) device.

Thanks David for the advice. Sadly the plugboard method you just described in more detail didn't work for me, nor with any of the four combinations I asked about above (i.e varying the device and authors vs author_sort.)

What DID work, strangely, was simply making sure there was no "and" in the author names.

______________________________________
EXAMPLES

Did NOT get detected on Forma, even when I used the methods above:

Smith, Johnson & Thomas
Smith & Thomas
("and" gets converted to "&" anyway, even if you write "and")

DID get detected on Forma, even BEFORE I used the methods above:

Smith et al
Smith, Johnson, Thomas
Smith; Johnson; Thomas
Smith + Johnson + Thomas

(NOTE: all included first names - skipped for brevity here)

[DNSB]

Originally Posted by LivresInOz

Did NOT get detected on Forma, even when I used the methods above:

Smith, Johnson & Thomas
Smith & Thomas
("and" gets converted to "&" anyway, even if you write "and")

DID get detected on Forma, even BEFORE I used the methods above:

Smith et al
Smith, Johnson, Thomas
Smith; Johnson; Thomas
Smith + Johnson + Thomas

(NOTE: all included first names - skipped for brevity here)

In calibre, you would need to have all your author names separated by a &.

I.E. Jacob Smith & Thomas Johnson & Sarah Thomas for the author name with the autogenerated author sort field being:
Smith, Jacob & Johnson, Thomas & Thomas, Sarah

Otherwise, the author names shown on your Kobo and in calibre will have issues matching.

Looking at one of my wife's books, author(s) is:

Annabelle Anders & Suzanna Medeiros & Tammy Andresen & K. J. Jackson & Tracy Sumner & Rachel Smith & Christine Murray Donovan & Darcy Burke & Robyn Dehart & Diana Bold & Heather Boyd & Nancy Yeager & Amy Quinton

while author sort is:

Anders, Annabelle & Medeiros, Suzanna & Andresen, Tammy & Jackson, K. J. & Sumner, Tracy & Smith, Rachel & Donovan, Christine Murray & Burke, Darcy & Dehart, Robyn & Bold, Diana & Boyd, Heather & Yeager, Nancy & Quinton, Amy

Anthologies are the reason that I truncate the authors and author sort to the first entry when generating the filename for my Kobo ereaders while leaving all the author names searchable.

Ebooks/{author_sort:sublist(0,1,&)}/{title} - {authors:sublist(0,1,&)}

[davidfor]

Originally Posted by LivresInOz

What worked for me and didn't, in trying to make sure books were detected as present on the Kobo (Forma) device.

Thanks David for the advice. Sadly the plugboard method you just described in more detail didn't work for me, nor with any of the four combinations I asked about above (i.e varying the device and authors vs author_sort.)

I don't see any "four combinations" in the above posts, so I am not sure what you mean. And it is possible you will need to resend books to solve some of these issues. But, reading the rest of the post I think I know what the problem is.

What DID work, strangely, was simply making sure there was no "and" in the author names.

______________________________________
EXAMPLES

Did NOT get detected on Forma, even when I used the methods above:

Smith, Johnson & Thomas
Smith & Thomas
("and" gets converted to "&" anyway, even if you write "and")

DID get detected on Forma, even BEFORE I used the methods above:

Smith et al
Smith, Johnson, Thomas
Smith; Johnson; Thomas
Smith + Johnson + Thomas

(NOTE: all included first names - skipped for brevity here)

Based on the above, the problem is that you are using what might be printed on the cover of a book, not the actual author names. And the different systems are interpreting theses in different ways.

For "Smith, Johnson & Thomas", calibre will consider that as two authors, "Smith, Johnson" and "Thomas". Calibre always uses the ampersand as the delimiter between names. When they are first entered it will also use other things, but, after that it will display them using the ampersand. And similarly, "Smith & Thomas" will be two authors, "Smith" and "Thomas". For all of the other examples, calibre will treat them as one author.

When the book is on the Kobo device, the author is split on commas. So, "Smith, Johnson & Thomas" becomes two authors, "Smith" and "Johnson & Thomas". "Smith, Johnson, Thomas" becomes three authors "Smith", "Johnson" and "Thomas". All the rest will be treated as a single author. The sorting will be based on the Surname in the author which is basically the last word. So, "Smith et al" should be sorted under "al" and most of the rest under "Thomas".

You didn't actually give any examples of the problems you had or otherwise my advice would have had some extra things in it. And that is to keep it simple for the authors. Calibre expects the author as the list of authors in "LN FN" format. Having middle names or initials works. You can fiddle with the author sort to achieve other things. But, not having the author like this breaks to many other things to make it worthwhile. Or allow it to work, as you have seen.

[DNSB]

As another source of information, you might want to check the author items under the calibre Tweaks section in Customizing calibre.

The default regex used by calibre to split multiple author names basically uses &, and and with as the possible seperators.

[LivresInOz]

Thanks all.
If I keep it simple by using commas between names, will a search for those names on the Kobo find them?

eg.
John Smith, Don Roberts, Mary McGee

The take-home of my msg above was that using an "&" (in author section) is what made the book not show up.

(Well specifically: an '&' before the last name. Havent tried '&' between all names)

[DNSB]

Originally Posted by LivresInOz

Thanks all.
If I keep it simple by using commas between names, will a search for those names on the Kobo find them?

eg.
John Smith, Don Roberts, Mary McGee

The take-home of my msg above was that using an "&" (in author section) is what made the book not show up.

(Well specifically: an '&' before the last name. Havent tried '&' between all names)

The & between the author names is needed by calibre. Unless you have set calibre to display LN, FN in the author field, there should be no commas. As far as I recall, your Kobo stock firmware will only read the first author name from the dc:creator fields in the .opf file however on the second connection (the send metadata phase), any other author names will be added to the ebook's database entry.

I.e. Calibre's Author(s) field contains John Ringo & Lydia Sherrer, series/index fields contains Into the Real [01].

On the first connect, the ebook is sent to your Kobo and imported. After the import, if you look at the ebook's library entry in Kobo's GUI, it will only show the first author name.

On the second connect, the metadata is added to the ebook's database entry. If you look at the ebook's library entry in Kobo's GUI, it now show both author's names and the series.

Edit: please note that I am using Metadata managment set to Automatic management. Otherwise on the second connect, I would have to manually start the metadata update.