Sorting Authors by Name is wrong in 6.4

[igorius]

@un_pogaz: So what a strange thing: I have to have 2 columns, Authors and Author_sort, which have both the same value in and a third column with author_sort inside, if i need both values (Authors and Authors_sort).
So sorry, but who had this strange idea? It seems extremely far away from real life! And imagine the big possibility of getting strange things with that 3 columns. I see only problems with this and no positive use.
Its like shooting to the sky to kill a mouse...
Why not make sorting the easy way? Why complicated? And therefore senseless (in my opinion). Or do you have a good explanation for that strange thing? Why can i "sort" the Authors column when i cannot sort like any other column?
What if i delete the column author_sort? Then sorting gets impossible?
So finally i see that that strange sorting "feature" creates more problems than use. There are lots of questions about this wrong sorting inside the forum, and not even one good answer about the WHY.

[chaley]

Originally Posted by igorius

What if i delete the column author_sort? Then sorting is impossible?

There is no built-in GUI column named author_sort. There is nothing you can delete other than going into metadata.db and changing the structure of the books table.

If you see an author_sort column (which apparently you do) then you created a custom column to show it.

[igorius]

Again, here the calibre-help:
https://manual.calibre-ebook.com/faq...es-and-sorting

Now coming to author name sorting:

When a new author is added to calibre (this happens whenever a book by a new author is added), calibre automatically computes a sort string for both the book and the author.

Authors in the Tag browser are sorted by the sort value for the authors. Remember that this is different from the Author sort field for a book.

By default, this sort algorithm assumes that the author name is in First name Last name format and generates a Last name, First name sort value.

Due to this explanation we can assume that the tag-browser is getting sorted by the Author_sort column.
My Authors are entered in the above mentioned way: FN LN. Described above.
That would mean, tag-browser should sort LN, FN which is done this way, even when its showing the names FN LN!
But the same help tells me "Remember that this is different from the Author sort field for a book."
So, which field is meant? Authors or Author_sort? Thats in no way clear! Tag browser is sorted by Author_sort but Author_sort is sorting which way??????
And Authors are sorted too which way? Like YOU said the same way as Author_sort. Why?
And that makes me assume when i change Author_sort to Author, ONLY the tag-browser is getting changed!
The Authors column is nowhere mentioned. So i have to assume that i can sort this column the way i want. Which i cannot do!
Anyone getting the strangeness of the whole thing?

So please would anyone explain please in clear and understandable words why its done the strange way?
Therefore we can also say that the help doesnt help in any way! That whole thing needs to be cleared for future audience.
So when i change Author_sort to Author, tag browser sorts authors the way FN LN and Authors the same way.
Therefore i need a third column which has Authors stored the way LN, FN!

Sorry, that whole thing is a big nonsense and in no way straight in terms of use!


@chaley: Thats wrong! Look at the pic from the DB! It contains Authors and Author_sort!

[theducks]

You have a Column? (AFAIK You only can see the one in Calibre in Manage Authors. All the rest is a single book, MDE view of the field)

what is the lookup name? (hover over column title)

[igorius]

@*theducks:
Lookup-name of authors is authors, lookup-name of author_sort is Author_sort

See the pics:

[chaley]

Originally Posted by igorius

@chaley: Thats wrong! Look at the pic from the DB! It contains Authors and Author_sort!

Sigh. I said there was no GUI column named author_sort, and there isn't. I also said you could go to the DB and change the structure of the books table. And BTW there is an (author_)sort column in the authors table that the tag browser uses.

Originally Posted by igorius

@*theducks:
Lookup-name of authors is authors, lookup-name of author_sort is Author_sort

No. The screen capture clearly shows that the lookup name is #author_sort. Note the '#' character. It is a custom column.

I give up. Find a different library organizer and have a good life.

[theducks]

Any lookup name that starts with a octothorpe (#) was made by YOU (or the person that customized your structure.
That also means there are ZERO settings that control it.
AND
You must maintain it (the contents).

BTW you just annoyed the developer that gave us custom columns and other features

[un_pogaz]

Originally Posted by igorius

So please would anyone explain please in clear and understandable words why its done the strange way?
Therefore we can also say that the help doesnt help in any way! That whole thing needs to be cleared for future audience.
Therefore i need a third column which has Authors stored the way LN, FN!

This strange behavior is due to the fact that some people sort authors differently from the way they are displayed.
The common case is that the book has as author "Isaac Asimov", but will be sorted in the bookstore/library shelves to "A" since it is based on the name of the author, not his first name.
The value "author_sort" allows this flexibility to display "Isaac Asimov" but to sort it into "Asimov, Isaac".
All this is confusing, I grant you, but it's because you are facing the default behavior of Calibre which may not be the one you find natural and obvious.
But this is also a strength of Calibre: As we have explained, there are many options to get the result YOU want.

You want the sort column to work with the values of the "Authors" column? Do as I say.

From what I've read of the FAQ, it describes pretty well, but it's not wrong.
Maybe this section needs some clarification, as the functionality is very complex and subtle. Shit, maybe just create a dedicated page to have room to add more examples of desired cases would help.
But as you have experienced, if anything is too complex and overhauled in the Manual, there is the forum.

Originally Posted by igorius

Therefore i need a third column which has Authors stored the way LN, FN!

Yep, but there are tricks to help you generate or fill it.

Also, for some reason, you have a column "Authors Sort", problem, it is not a Calibre column. So I can understand that this presence has misled you, but this column is only visual and informative, you can't change it. You probably added it, or someone else. It shows you how Calibre really works.

[BetterRed]

If you set the Author sort name algorithm tweak to 'copy' and then go into Manage Authors and hit the Recalculate all author sort values button you'll get what you want - Mark Twain will be amongst the M's rather than the T's.

It may take a minute or so to take effect while the database Authors table is rewritten.

The default settings are the way they are because calibre is an e-book LIBRARY manager, not an ebook FILE manager. Bricks & mortar libraries and book stores shelve books by author last name (LN, FN), which is what the calibre default settings model.

This won't be the last feature of calibre that will need adjustment to its settings and customisations to meet your specific idiosyncrasies. So, if you aren't prepared to spend a couple of minutes changing a tweak and clicking a button to get authors sorted the way you want (FN LN) perhaps you should use another ebook library manager.

BR

[igorius]

I love it!
So till now we were discussing a philosophical library searching.
But now lets build a real world library...
We install calibre, we do NOT touch any preferences to get all the same results. But we add the custom column author_sort.
We have a metadata db, which contains 2 columns: authors and sort.
Now we import a book with only one author and we enter all data without any error, because we are perfect.
We get: (according the calibre help!)
GUI.............................................DB
Authors: .........."Annie Author"......: Authors: ..."Annie Author" -> Author(GUI) is a reflection of Author (DB)
Author_sort:....."Author, Annie"......: Sort: ........"Author, Annie" -> Author_sort (GUI) is a reflection of Sort (DB)
So far so good!
Now lets look at sorting:
Author_sort (GUI) is a reflection of Sort (DB)
Authors (GUI) is an abstraction of Sort (DB), so sorting goes "wrong", its not natural sorting.
To get a natural sorting of Authors (GUI), we make an abstraction of the reflection of Author_sort (GUI), which is a reflection of sort (DB): We create a new column (GUI), which reflects this abstraction of the reflection.
Now sorting Authors (GUI) is the reflection of an abstraction of a reflection.
With our newly imported book we get:
Author_sort (GUI): ..."Author, Annie"
New column (GUI): ..."Author, Annie"
Authors (GUI): ........."Annie Author"
Anything good!
Now we fill up the Library with millions of books and fill in wrong data for 1 Author, like in real life...
Authors (GUI): ......."Annie Author" .................... gets -> Authors (DB): "Annie Author"
Author_sort (GUI): ."Author, Annew" ................. gets -> Sort (DB): ......" Author, Annew"
New column (GUI): "Author, Annew" (we know its the reflection of an abstraction of a reflection!)
Due to the fact of this Abstraction of the reflection of the previous reflection we loose our Author and will find her never again, because we have no direct reflection of our Authors (DB) to Authors (GUI) -> speaking of sorting them.
The data is still here, but not any more visible in terms of sorting.
If we had a reflection of the Authors (DB) to Authors (GUI), we would see directly our mistake we made when entering the names.
This is my problem with the actual kind of sorting -> i had exactly this case!
Now the developers of calibre should tell us, why the sorting of authors(GUI) is not directly reflected with the Authors (DB).
And by the way, no one will find out without help the fact that you have to make that complicated abstraction/reflection-thing!

And we are using a simple name, imagine if we have a name like "Prof. Dr. Dr. HC Igor Maria Duke of GoHome IV"

And what i read all the time her (i have asked various times) is:
It was never intended, which means to me: "We have no clue, its like it is"
You should add a new composite column... which means to me: "Argh, i cannot explain, but i have a kind of remedy"
Both answers are what i heard when learning my profession: "We always did it this way" and "You have to do what we tell you".
When i begun studying and working, i learned that all times that i nearly never have the easy way, i ask why and find better solutions!

So again: Who can explain that "feature" that i possibly (and others too) get the fundamental idea behind this (to me) obvious nonsense of sorting authors!

At last all your answers help me to get what i want, but at last no one can obviously tell why these things are like they are, only the devs who invented that thing.

Hopefully one can explain it!

I will thank you all for your patience and really good explanations for my annoying questions!

[BetterRed]

@other responders

I have several libraries that sort authors differently - e.g. my Books library sorts LN, FN because the Author sort algorithm tweak is set to 'invert', whereas my Media Library sorts FN LN because the Author sort algorithm tweak is set to 'copy'. I don't have any Author related composite columns, if I want to see (or change) what's in author_sort I use the Manage Authors list.

I've been doing it this way for a over decade. Prior to Dalton ST providing Per Library tweaks of the Author sort algorithm I got there via multiple installs of Calibre Portable.

ETA: as I edit metadata in the book list I instinctively glance down at the book details to check what I just entered each time I hit tab to move to the next cell. I display title_sort and author_sort in book details, so I also spot any anomalies there - e.g. unhyphenated double-barrel surnames that need 'fixing' in Manage Authors.

BR

[DNSB]

Originally Posted by igorius

I love it!
So till now we were discussing a philosophical library searching.
But now lets build a real world library...
We install calibre, we do NOT touch any preferences to get all the same results. But we add the custom column author_sort.
We have a metadata db, which contains 2 columns: authors and sort.
Now we import a book with only one author and we enter all data without any error, because we are perfect.
We get: (according the calibre help!)
GUI.............................................DB
Authors: .........."Annie Author"......: Authors: ..."Annie Author" -> Author(GUI) is a reflection of Author (DB)
Author_sort:....."Author, Annie"......: Sort: ........"Author, Annie" -> Author_sort (GUI) is a reflection of Sort (DB)
So far so good!
Now lets look at sorting:
Author_sort (GUI) is a reflection of Sort (DB)
Authors (GUI) is an abstraction of Sort (DB), so sorting goes "wrong", its not natural sorting.
To get a natural sorting of Authors (GUI), we make an abstraction of the reflection of Author_sort (GUI), which is a reflection of sort (DB): We create a new column (GUI), which reflects this abstraction of the reflection.
Now sorting Authors (GUI) is the reflection of an abstraction of a reflection.
With our newly imported book we get:
Author_sort (GUI): ..."Author, Annie"
New column (GUI): ..."Author, Annie"
Authors (GUI): ........."Annie Author"
Anything good!
Now we fill up the Library with millions of books and fill in wrong data for 1 Author, like in real life...
Authors (GUI): ......."Annie Author" .................... gets -> Authors (DB): "Annie Author"
Author_sort (GUI): ."Author, Annew" ................. gets -> Sort (DB): ......" Author, Annew"
New column (GUI): "Author, Annew" (we know its the reflection of an abstraction of a reflection!)
Due to the fact of this Abstraction of the reflection of the previous reflection we loose our Author and will find her never again, because we have no direct reflection of our Authors (DB) to Authors (GUI) -> speaking of sorting them.
The data is still here, but not any more visible in terms of sorting.
If we had a reflection of the Authors (DB) to Authors (GUI), we would see directly our mistake we made when entering the names.
This is my problem with the actual kind of sorting -> i had exactly this case!
Now the developers of calibre should tell us, why the sorting of authors(GUI) is not directly reflected with the Authors (DB).
And by the way, no one will find out without help the fact that you have to make that complicated abstraction/reflection-thing!

And we are using a simple name, imagine if we have a name like "Prof. Dr. Dr. HC Igor Maria Duke of GoHome IV"

And what i read all the time her (i have asked various times) is:
It was never intended, which means to me: "We have no clue, its like it is"
You should add a new composite column... which means to me: "Argh, i cannot explain, but i have a kind of remedy"
Both answers are what i heard when learning my profession: "We always did it this way" and "You have to do what we tell you".
When i begun studying and working, i learned that all times that i nearly never have the easy way, i ask why and find better solutions!

So again: Who can explain that "feature" that i possibly (and others too) get the fundamental idea behind this (to me) obvious nonsense of sorting authors!

At last all your answers help me to get what i want, but at last no one can obviously tell why these things are like they are, only the devs who invented that thing.

Hopefully one can explain it!

I will thank you all for your patience and really good explanations for my annoying questions!

This seems to be the first time you have mentioned incorrect data in the author_sort field. One has to wonder how this is done other than by a manual edit, i.e. PEBKAC error. The normal procedure when using the MDE is to use calibre to automatically generate the author_sort from the author

The phrase that comes to mind is Garbage In, Garbage Out. You have put incorrect data into the database, the GUI reads the database and reflects that incorrect data. Sadly, calibre is unable to read your mind and automatically correct your errors.

This contrasts sharply with your original complaint that calibre by design sorts authors by using the author-sort field which is stored as last name, first name, initials which is how calibre sorted from the first version I ran. This matches what I have seen in every physical and online library I've searched for books in. When I search for a book by Robert Asprin, I look under A for Asprin and not R for Robert.

Edit: Did you try the suggestion of selecting all your books, going to edit metadata in bulk, select the automatically set author sort and hitting OK? That should correct any errors in the author_sort field though it will not fix any errors in the author field.

Edit2: Have you tried any of the suggestions made in this thread? Or are you simply repeating your question in the hopes someone will wave their magic wand and fix your corrupted metadata?

[un_pogaz]

Yep, the ePub can already contain an "author_sort" value and Calibre will import it as is, so be careful with it.

A simple solution is to use Action Chains and install the script Bulk Sort. This solution will rebuild all author_sorts based on the DB value.

A more advanced solution with template function:
@BetterRed you may be interested

Create the template function valid_author_sort()

program: ifelse($author_sort, author_sorts(' & '), '1', '0')

when the ifelse() allows a sensitive comparison to the text:

def evaluate(self, formatter, kwargs, mi, locals, x, y, yes, no):
    if str(x) == str(y):
        return yes
    else:
        return no

Personally, I use it to color the columns in red when "author_sort" is invalid, and as well as a saved search for quick check ma library. No need to have it in book details, one line saved.

template:"program:valid_author_sort()#@#:t:0"

And this is much faster than launching Quality Check, no need a space scraft for travel across the street.

[chaley]

Originally Posted by un_pogaz

when the ifelse() allows a sensitive comparison to the text:

def evaluate(self, formatter, kwargs, mi, locals, x, y, yes, no):
    if str(x) == str(y):
        return yes
    else:
        return no

@un_pogaz: Please see https://www.mobileread.com/forums/sh...d.php?t=348976.

[BetterRed]

@un_pogaz - the author_sort can be valid according to the algorithm used (e.g. 'invert'), but that does not mean it is correct. Examples:

If I enter "Madeleine St John" as Author then the resultant author_sort will be "John, Madeleine St" which is wrong. The correct value is "St John, Madeleine".

Similarly, "Gabriel García Márquez" will result in author_sort value of "Márquez, Gabriel García" but the correct value is "Garcia Marquez, Gabriel"

In both instances I changed the author_sort value in Manage Authors at the time the first book for those authors was being added and it has been correct ever since.

IMO the formation of sort names can never be fully automated - there will always be exceptions - and calibre allows for that, and I prefer to do it at the point of creation.

BR