Author Sort is messing things up

[DrJanet]

Hi,

Thank you for Calibre - it's like iTunes for ebooks; you've done a great implementation. Unfortunately, the actions of the Author Sort field is severely limiting the app's usefulness for me. I have read in earlier posts that it cannot be deactivated (too bad), but, is there any way to modify or over-ride it? Or, can we just chose to sort on the authors field directly?

Here's the problem: I have imported over 2500 books, with a few thousand more to go. The file names all had the format of AuthLastName, FirstName & 2ndAuthor - Series - Title. So they alphabetized by the authors' last name.
ex. Bradley, Marion Zimmer & Ross, Deborah J - (Darkover Clingfire Trilogy 1) - The Fall of Neskaya (.rtf v1.0).rar

When imported, Calibre correctly breaks the filename down to put the authors and series into the author field and the title into the title field, which is fine. (Of course I would love to be able to set - as a delimiter and direct Caliber to put the series, and series order into other fields, but that's for the future wishlist; see the MS Excel UI for this ;o) So I end up with the author's name looking like this: Bradley, Marion Zimmer & Ross, Deborah J - (Darkover Clingfire Trilogy 1).

Author Sort seems to assume that the text string following the last space is really the authors' last name, and that the rest is the FirstName, so it ends up as 1), Bradley, Marion Zimmer & Ross, Deborah J - (Darkover Clingfire Trilogy totally screwing up the alphabetizing order which was correct to begin with. Checking the Swap author firstname and lastname box in the Add prefs seems to have no effect on this behaviour. Worse, when I edit the metadata by hand (which isn't really viable with this many books, anyway), and paste the authors name into the author sort field, it reverts back to the original incorrect entry. The only fix is to remove everything from the author field (there is no place to add additional authors, btw) and swap the first and last name. It's just not feasible with this many books.

So, please tell me there's another fix for this, or will be in the future. Thanks.

[DrJanet]

BTW, I'm using the Mac version.

[itimpi]

Normnally the Author field is expected to be in the form FirstName LastName. When I do this the Author Sort field gets set as expected.

Until Calibre treats FirstNames(s) and LastName as distininct fields I do not think that a tidy solution that suits everyone is possible.

[kovidgoyal]

You can add multiple authors by using the & delimiter and edit the author sort field directly using the edit meta information dialog. You can also customize how metadata is read from filenames view Preferences->Add/Save

[Starson17]

Originally Posted by itimpi

Until Calibre treats FirstNames(s) and LastName as distininct fields I do not think that a tidy solution that suits everyone is possible.

As of version 0.6.31 there is a checkbox in configuration for ADD/SAVE that swaps the authors firstname and lastname. It works great for authors without a middle name or initial. There are still problems with those.

[theducks]

Originally Posted by DrJanet

Hi,

Thank you for Calibre - it's like iTunes for ebooks; you've done a great implementation. Unfortunately, the actions of the Author Sort field is severely limiting the app's usefulness for me. I have read in earlier posts that it cannot be deactivated (too bad), but, is there any way to modify or over-ride it? Or, can we just chose to sort on the authors field directly?

Here's the problem: I have imported over 2500 books, with a few thousand more to go. The file names all had the format of AuthLastName, FirstName & 2ndAuthor - Series - Title. So they alphabetized by the authors' last name.
ex. Bradley, Marion Zimmer & Ross, Deborah J - (Darkover Clingfire Trilogy 1) - The Fall of Neskaya (.rtf v1.0).rar

When imported, Calibre correctly breaks the filename down to put the authors and series into the author field and the title into the title field, which is fine. (Of course I would love to be able to set - as a delimiter and direct Caliber to put the series, and series order into other fields, but that's for the future wishlist; see the MS Excel UI for this ;o) So I end up with the author's name looking like this: Bradley, Marion Zimmer & Ross, Deborah J - (Darkover Clingfire Trilogy 1).

Author Sort seems to assume that the text string following the last space is really the authors' last name, and that the rest is the FirstName, so it ends up as 1), Bradley, Marion Zimmer & Ross, Deborah J - (Darkover Clingfire Trilogy totally screwing up the alphabetizing order which was correct to begin with. Checking the Swap author firstname and lastname box in the Add prefs seems to have no effect on this behaviour. Worse, when I edit the metadata by hand (which isn't really viable with this many books, anyway), and paste the authors name into the author sort field, it reverts back to the original incorrect entry. The only fix is to remove everything from the author field (there is no place to add additional authors, btw) and swap the first and last name. It's just not feasible with this many books.

So, please tell me there's another fix for this, or will be in the future. Thanks.

Calibre does not parse out Series and series_index from file names by default.
You probably need to edit out by hand and enter additional fields.
Series: Darkover Clingfire Trilogy
Book: 1

There is even less standardization in title naming with series than there is for serial books.
Star Wars: The Return of the Jedi or Return of the Jedi?
Just try and do a Meta-Data fetch on some of these compound titles.

Database Normalization is a bear to accomplish

[DrJanet]

Originally Posted by kovidgoyal

You can add multiple authors by using the & delimiter and edit the author sort field directly using the edit meta information dialog. You can also customize how metadata is read from filenames view Preferences->Add/Save

Thanks for your reply. I see what you are saying about multiple authors. Of course with these authors one still has to tweak it:

Authors: Bradley, Marion Zimmer & Lackey, Mercedes
Author Sort: Zimmer, Bradley, Marion & Mercedes, Lackey,

This is manageable, but my main problem is with the other several thousand books I have. They are, unfortunately, already named in the format I have mentioned. Swapping the 1st & last name wouldn't work due to the extraneous info in their titles, and bringing them in without the sway yields correctly parsed data in the title & authors fields. Then author sort comes along and screws it all up.

So, I'll ask again, is there any way to modify, work around it or over-ride it, without having to edit each entry manually? If not, is a fix for this something we are likely to see in the future? I don't mind paying for such a useful application.

[DrJanet]

Originally Posted by itimpi

Until Calibre treats FirstNames(s) and LastName as distininct fields I do not think that a tidy solution that suits everyone is possible.

Yes, that's why I would want to at least disable it, or have the choice to sort on the Authors field directly. Swapping the names is even messier in my case due to the extra info in the file names. Thanks.

[kovidgoyal]

I'm afraid manual correction is your only option at this point.

[theducks]

Originally Posted by DrJanet

Thanks for your reply. I see what you are saying about multiple authors. Of course with these authors one still has to tweak it:

Authors: Bradley, Marion Zimmer & Lackey, Mercedes
Author Sort: Zimmer, Bradley, Marion & Mercedes, Lackey,

This is manageable, but my main problem is with the other several thousand books I have. They are, unfortunately, already named in the format I have mentioned. Swapping the 1st & last name wouldn't work due to the extraneous info in their titles, and bringing them in without the sway yields correctly parsed data in the title & authors fields. Then author sort comes along and screws it all up.

So, I'll ask again, is there any way to modify, work around it or over-ride it, without having to edit each entry manually? If not, is a fix for this something we are likely to see in the future? I don't mind paying for such a useful application.

So, how many wrong name order Author combinations (assuming the Extraneous stuff got removed)?
Sort your Library by the Author column. Highlight the same names (desired to be exactly the same) and use Bulk Meta-data edit to fix the group in 1 step, only 300 to go.

[DrJanet]

Originally Posted by theducks

So, how many wrong name order Author combinations (assuming the Extraneous stuff got removed)?
Sort your Library by the Author column. Highlight the same names (desired to be exactly the same) and use Bulk Meta-data edit to fix the group in 1 step, only 300 to go.

Lots ;o) Sorting by Author is a non-starter since Author Sort totally mixes them up. So far, if I decide to use Calibre and invest the time, I would have to do a search on each author, then apply bulk edits. The kicker is that my books are already in the proper order by author in the folders on my drive, so the cost/benefit ratio of Calibre is close to 1. Thanks.

[DrJanet]

Originally Posted by theducks

Calibre does not parse out Series and series_index from file names by default.

Database Normalization is a bear to accomplish

Tell me about it! i work in healthcare IT and data normalization is one of the biggest stumbling blocks to creating an electronic medical record. Thanks for your reply.

[DrJanet]

Originally Posted by kovidgoyal

I'm afraid manual correction is your only option at this point.

Oh, too bad. Do you have any plans for continuing to work on Calibre and possibly modifying this in the future? If so, I have some other UI suggestions I would be happy to share with you, if you like. It is really a very nice piece of software that is certainly worth improving upon. Thank you.

[Starson17]

Originally Posted by DrJanet

Oh, too bad. Do you have any plans for continuing to work on Calibre and possibly modifying this in the future? If so, I have some other UI suggestions I would be happy to share with you, if you like. It is really a very nice piece of software that is certainly worth improving upon. Thank you.

You have several options:

First, there is manual changing, but that's not viable.

Second, you can rewrite the open code.

Third, you can bulk preprocess your books to rename them to the way that calibre prefers (firstname lastname). That's probably the easiest for you. I suggest Booksorter. I've also used batch files to do this job. I made a temporary copy of the books, bulk renamed and then imported them.

Fourth, you can use calibredb to output the opf file metadata, process it with SED to fix your author names, and use calibredb again to bring it back in. I did this on thousands of books, and it's slow, but it works.

BTW, you wrote"Of course I would love to be able to set - as a delimiter and direct Caliber to put the series, and series order into other fields, but that's for the future wishlist."

I'm not sure if you realize it, but it's quite easy to automatically put the series and series_index into other fields using the regex expression in Add/Save. A search on "regex" here will give you expressions to match your current filename format and bring in those fields correctly.

Good Luck.

[theducks]

Originally Posted by DrJanet

Lots ;o) Sorting by Author is a non-starter since Author Sort totally mixes them up. So far, if I decide to use Calibre and invest the time, I would have to do a search on each author, then apply bulk edits. The kicker is that my books are already in the proper order by author in the folders on my drive, so the cost/benefit ratio of Calibre is close to 1. Thanks.

You mis-understood me.
Sort the Displayed list to show side by side by side.
This changes nothing except the display order on the screen.
Now you can see the similars. I fix Publisher info this way
Baen Books, Baen Pub., Baen Publishers, Baen. All get Highlighted and set to my standard: Baen Books. in a few clicks.