New to Calibre, having some difficulties

[Merischino]

Without including the entire list, here is a snippet of my list. Please note that in all cases but one (Anne McCaffrey), the Author and the Author Sort fields are populated with the exact same data (example I have copied and pasted "Batho, Michele" into both the "Author" and the "Author Sort" fields for Michele Batho.) Only in the case of Anne McCaffrey are some records populated with "Anne McCaffrey" as the author and "McCaffrey, Anne" as the author-sort.) This discrepancy is because I am troubleshooting. For brevity I am skipping over a lot of records to show the pattern of discrepancies. This by no means includes all discrepancies.

1. Aesop
2. Norton, Andre
3. McCaffrey, Anne
28. Asimov, Isaac
30 Austen, Jane
33 Michele Batho
35 Brown, Dan
58 Doyle, Sir Arthur Conan
59 Cervantes Saavedra, Miguel de
60 Adams, Douglas
72 Doyle, Sir Arthur Conan
75 Eddings, David
123 McCaffrey, Anne
131 Anne McCaffrey
147 McCaffrey, Anne
252 Stableford, Brian

Now, after having moved my calibre library, things are starting to make a *little* bit more sense in this list than they have in the past. An argument could be made that sorting is now happening on the first name. Sometimes. then, sometimes, for no apparent reason, it switches to sortying by last name. (Doyle? Cervantes?) or by middle name maybe (the "de" in Miguel de Cervantees Saavedra maybe?)

If I have "Adams, Douglas" entered for both Author and Author Sort fields, then shouldn't "Adams, Doglas" be sorting on the A of Adams, and not on the D of Douglas? If It is sorting on the D of Douglas, shouldn't it follow that it would sort also on the o after that D and place itself AFTER the Da in Eddings, David?

If Sir Arthur Conan Doyle is entered into both the Author and the Author Sort fields in the GUI metadata as "Doyle, Sir Arthur Conan" the does it not follow that it should be sorting on the D in Doyle, not the S of Sir or the A of Arthur or the C of Conan, and that it should sort all Doyle, Sir Arthur Conan entries the same way and in the same place in the list? Why are the two "Doyle" entries not together?

If Michele Batho's metadata is entered into Calbre as Batho, Michele in all cases, why does she show up in this list as "Michele Batho"?

If Calibre is sorting on first initial of first name, why is Brian Stableford all the way down at #352?

If there are only two ways in which Anne McCaffrey is entered into Metadata in Calbire (namely, 1 as "Anne McCaffrey" as author and as "McCaffrey, Anne" as author-sort and 2 as "McCaffrey, Anne" as author and as "McCaffrey, Anne" as author-sort, with both scenarios establishing author sort in the same way, why are there more than 4 different ways that she is sorting? Based on the metadata entries, she should sort only one way, by the last name. In practice, she sorts by last name, by first name, and by who knows what order when entries appear somewhere else altogether in the list

[Worldwalker]

Originally Posted by Pagali

Sorry to digress from the OP's problem, but... how do you do that? I'm loading books onto a Sony Reader PRS600, and I'd rather not have the series showing up there as collections.

Preferences
Plugins
Device Interface Plugins
Sony Device Interface
Customize Plugin (button)
remove "series" from the bottom text box

[Merischino]

I should add that the desired end-state is that in my view, Calibre should uniformly sort based on the actual metadata entered into the Author-Sort field unless and ONLY unless there is nothing in the Author-Sort field, in which case it should uniformly sort on what is in the Author field.

This expecation would work like this:

Metdata for 3 authors entered as follows

Author Field Author-Sort field
Anne McCaffrey McCaffrey, Anne
McCaffrey, Anne McCaffrey, Anne
Anne McCaffrey ***blank****
Isaac Asimov Asimov, Isaac
Robert A Heinlein Heinlein, Robert A

Assuming the above data, and assuming that Calibre is set up so that author-sort defines the sort, then this list should have only one discrepancy, and that is a single one of the Anne McCaffrey items, the one where there is nothing defined in "author-Sort". With the above metadata, the Author list when sorted SHOULD look like this"

Anne McCaffrey
Asimov, Isaac
Heinlein, Robert A
McCaffrey, Anne
McCaffrey, Anne
McCaffrey, Anne

Where there is one entry for each entry in the underlying database, and where the record appears in the list as it is entered in the Author Sort field, in alphabetical order based on the initial character of that field. Only in the case of the author-sort field being empty should there be a discrepancy. **Actually, if I understand the way calibre works correctly, even in the above example the "anne mccaffrey" entry should automatically default to "McCaffrey, Anne" because that's the way the author-sort field populates when you check the box telling it to do so.

I do not know why in actuality, the records SORT on the first name, but appear in the list with the last name first. I can't think of any logical reason why that should be. Especially if the metadata is entered where the first name is in the field after the last name. So, "Brown, Dan" appearing in the list as "Brown, Dan" but sorting on the D, not the B, does not make sense to me.

[Starson17]

Originally Posted by Merischino

Without including the entire list, here is a snippet of my list. Please note that in all cases but one (Anne McCaffrey), the Author and the Author Sort fields are populated with the exact same data

I've looked at your db, and there are lots of entries where this is not what I see.

(example I have copied and pasted "Batho, Michele" into both the "Author" and the "Author Sort" fields for Michele Batho.)

I see "Batho, Michele" in author_sort and "Michele Batho" in author. They sort correctly under "B"

Only in the case of Anne McCaffrey are some records populated with "Anne McCaffrey" as the author and "McCaffrey, Anne" as the author-sort.)

I see the opposite - "Anne McCaffrey" as the author_sort and "McCaffrey, Anne" as the author.


Are you selecting a record, pressing "E" to edit the metadata (or right clicking and choosing "Edit metadata" to see the author_sort contents?

Sometimes. then, sometimes, for no apparent reason, it switches to sortying by last name. (Doyle? Cervantes?) or by middle name maybe (the "de" in Miguel de Cervantees Saavedra maybe?)

The contents of author_sort is: "de, Cervantes Saavedra, Miguel" so it sorts under "d"

If I have "Adams, Douglas" entered for both Author and Author Sort fields, then shouldn't "Adams, Doglas" be sorting on the A of Adams, and not on the D of Douglas?

But, you don't. You have "Douglas, Adams," in author_sort.

If It is sorting on the D of Douglas, shouldn't it follow that it would sort also on the o after that D and place itself AFTER the Da in Eddings, David?

It would, but you have "Eddings, David" in author_sort, so it's sorting on "E"

If Sir Arthur Conan Doyle is entered into both the Author and the Author Sort fields in the GUI metadata as "Doyle, Sir Arthur Conan" the does it not follow that it should be sorting on the D in Doyle, not the S of Sir or the A of Arthur or the C of Conan, and that it should sort all Doyle, Sir Arthur Conan entries the same way and in the same place in the list? Why are the two "Doyle" entries not together?

You don't have the same thing in author_sort for both. You have one record where "Conan" is first in author_sort and another where "Doyle" is first. Taht's wwhy tehy aren't together.

I could go on, but it's all sorting correctly - you just have a mess in your author_sort.

[Starson17]

Originally Posted by Merischino

I do not know why in actuality, the records SORT on the first name, but appear in the list with the last name first. I can't think of any logical reason why that should be. Especially if the metadata is entered where the first name is in the field after the last name. So, "Brown, Dan" appearing in the list as "Brown, Dan" but sorting on the D, not the B, does not make sense to me.

I don't want to belabor the point, but the records display what is in "author" and sort by what is in "author_sort" To help you fix it, I have to make sure you are looking at the content of the two fields. For some reason, you either are not seeing what I see, or you aren't looking at the right location.

To further explain, here is how Calibre works: When you add a book, you tell Calibre the author name. Calibre then tries to figure out the author_sort name. It's not very smart about this and just sticks the last part of the name at the front, separated by a comma. From then on, it will not change the author_sort. If it isn't sorted as you want, you need to change author_sort. If the author name isn't showing up correctly, you need to change that. Changing one does not change the other.

If you prefer to use Lastname, Firstname, there's a tweak that will prevent this name reversal or control it when Calibre enters the author_sort field. If you don't use the Tweak, and tell Calibre the name is "Brown, Dan," Calibre will just enter "Dan, Brown" into the author_sort field and sort on D.

[Merischino]

Originally Posted by Starson17

I don't want to belabor the point, but the records display what is in "author" and sort by what is in "author_sort" To help you fix it, I have to make sure you are looking at the content of the two fields. For some reason, you either are not seeing what I see, or you aren't looking at the right location.

To further explain, here is how Calibre works: When you add a book, you tell Calibre the author name. Calibre then tries to figure out the author_sort name. It's not very smart about this and just sticks the last part of the name at the front, separated by a comma. From then on, it will not change the author_sort. If it isn't sorted as you want, you need to change author_sort. If the author name isn't showing up correctly, you need to change that. Changing one does not change the other.

If you prefer to use Lastname, Firstname, there's a tweak that will prevent this name reversal or control it when Calibre enters the author_sort field. If you don't use the Tweak, and tell Calibre the name is "Brown, Dan," Calibre will just enter "Dan, Brown" into the author_sort field and sort on D.

hmmm.

Here's what I think:
a) My correcting the metadata isn't getting "saved" (however it is that one "saves" thbeir changes
b) My having a corrected (according to my beliefs, which granted could be faulty) database when I posted the first item in this thread, and my having continued making changes and what have you based on what I've read as a result of posting on this thread, including moving my library for example before posting my metadata.db file and
c) something I don't yet understand which could potentially make my typing "e" to edit the metadata, making standardized entries to "author" and "author sort" fields and expecting that my changes both register and are retained (either I'm going into the data "the wrong way" or what I'm doing "isn't taking" or something else is going on.

For example, if I go in right now as I write this and I find Don Quixote as a record in my database, hit "e" to enter the metadata editing window, what I see as "author-sort" is the same as what is entered for "author", namely Cervantes Saavedra, Miguel de

Now, I might want to enter ti a different way and have it sort on something other than the "d". but that isn't what my problem is. It's that, no matter what changes I make to the database, it keeps reverting itself back to the utter chaos it was at the moment immediately after I added all the files.

so that, Anne McCaffrey is entered into the Author and Author-Sort fields in a multiplicity of ways. And as a result, her name shows up in sorts in a hundred different ways. And, me making changes to the metadata does not appear to have much impact on that chaos.

So... clearly, whatever I'm doing ain't cutting it. I wish it were just a simple cut and dried user-error issue because I can learn how to fix my own behavior.

[Merischino]

Originally Posted by Starson17

I've looked at your db, and there are lots of entries where this is not what I see.



I see "Batho, Michele" in author_sort and "Michele Batho" in author. They sort correctly under "B"



I see the opposite - "Anne McCaffrey" as the author_sort and "McCaffrey, Anne" as the author.


Are you selecting a record, pressing "E" to edit the metadata (or right clicking and choosing "Edit metadata" to see the author_sort contents?



The contents of author_sort is: "de, Cervantes Saavedra, Miguel" so it sorts under "d"



But, you don't. You have "Douglas, Adams," in author_sort.



It would, but you have "Eddings, David" in author_sort, so it's sorting on "E"




You don't have the same thing in author_sort for both. You have one record where "Conan" is first in author_sort and another where "Doyle" is first. Taht's wwhy tehy aren't together.

I could go on, but it's all sorting correctly - you just have a mess in your author_sort.

It is interesting that what you see in the "metadata.db" file is corresponding with how things sort, however what I see in the calibre GUI does not in any way correspond to what you are seeing in the metadata.db file.

[Merischino]

Originally Posted by Starson17

I don't want to belabor the point, but the records display what is in "author" and sort by what is in "author_sort" To help you fix it, I have to make sure you are looking at the content of the two fields. For some reason, you either are not seeing what I see, or you aren't looking at the right location.

To further explain, here is how Calibre works: When you add a book, you tell Calibre the author name. Calibre then tries to figure out the author_sort name. It's not very smart about this and just sticks the last part of the name at the front, separated by a comma. From then on, it will not change the author_sort. If it isn't sorted as you want, you need to change author_sort. If the author name isn't showing up correctly, you need to change that. Changing one does not change the other.

If you prefer to use Lastname, Firstname, there's a tweak that will prevent this name reversal or control it when Calibre enters the author_sort field. If you don't use the Tweak, and tell Calibre the name is "Brown, Dan," Calibre will just enter "Dan, Brown" into the author_sort field and sort on D.

I guess I'd like to ask what this tweak is. Or whether using it, or not using it, would affect the behavior of the sorting for those of us (like me!) who want the sorting and the appearance of the name in that sorted list to be the same.

eg Author: McCaffrey, Anne
Author-sort: McCaffrey, Anne
Appearance in sorted list looks like: McCaffrey, Anne

Not sure whether this is even relevant given that my metadata.db file does not reflect my changes to my metadata, but assuming we can fix that and it does ultimately reflect my changes, would my entering the exactly same data into both of those two fields cause a conflict?

Up til now, the above scenario is what I have been trying to drive towards. So, forgetting for a moment what shows up in my metadata.db, if all data in the "author-sort" field is user-entered a specific way, will Calibre perform based on what's actually user-entered into that field, or will it supercede the actual entry as if there was nothing entered in there at all? (e.g, Does calibre assume in all cases that what it would have auto-entered is what's actually in there?)

Not trying to come up with a conspiracy theory. Just trying to figure out if, after completely deleting and clean-installing everything, and starting over from scratch, whether or not spending time entering data into the "author sort" field is a worthwhile effort.

[Starson17]

Originally Posted by Merischino

It is interesting that what you see in the "metadata.db" file is corresponding with how things sort, however what I see in the calibre GUI does not in any way correspond to what you are seeing in the metadata.db file.

What I see in the GUI exactly matches what I see in the metadata.db. I'm doing all my tests with the GUI - exactly as you are using it. Are you looking at the author_sort field by pressing "E" with the book selected?

[Merischino]

short answer: yes.

Long answer:
to "correct" the metadata for all of Anne McCaffrey's 105 works (in my database), i perform a number of different procedures, all of which include the steps of selecting the works(s) in a calibre GUI list, hitting "e" to open what I will call the 'metadata editing window" and then depending on whether there are multiple items or a single item selected, making changes to the Author and the Author Sort fields, if it's an actual novel downloading metadata such as the ISBN, to enable doanloading of the cover. Once all that is done, I close out that metadata editing window, I hit "c" to "convert" the file back into the same format it was when I began.

So, for Anne McCaffery's "Power Lines", which may show up at first in my library as "Power Lines" by "Power Lines" showing up in a list of authors sorted as "Power Lines" (note, not reversed)

I would select that item, hit "e"
In the dialog box that opens, I would change the Author to "McCaffrey, Anne" and then select that text and tab over to the next field for "author sort" and hit command-V to paste the duplicate text into the second field.

I would then possibly make other edits buyt for now we are focusing on the author and author sort issues.

After making those two fields display the same, desired, text, I would click "ok"

and expect that my library would now sort according to the new information, irrespective of whether or not I have overwritten the existing files with the new data. E.g., I would expect "Power Lines" on the left under "Title" to be the same, "Power Lines" to the right under author to have changed, and to read "McCaffrey, Anne", and if I removed the search query and sorted the total library on "author" I would expect the book "Power Lines" to appear in the "M" section for McCaffrey, Anne and not in the "A" section, and not to show "Anne McCaffrey" in either the "a" or the "m" section, but to show "McCaffrey, Anne" in the "m" section of that list.

Now, I do not know when my changes to the metadata within the GUI get reflected in the Metadata.db. Whether that happens instantaneously or only if/unless I convert (or re-convert, since it's going to the same format) or (ugh) "save to disk".

But, only one thing can I say is true:
I have put a lot of time and effort into changing the metadata, by hitting "e" and editing the metadata in that window, to reflect what I have b een posting here as what I believed to be in my database.

Namely, that all my author fields should have the same exact entry as the author-sort field.

What happens with it after I hit "e" and make edits, I cannot speak to.

And, again I thank you for your help and apologize for my frustration, which I know must be coming through. It's not directed at you.

[jackie_w]

@Merischino
I'm a bit nervous about poking my nose in here but I'm going to try anyway...

Can we go back to basics and try a very simple task.

1. Choose a book which has incorrect metadata.
   
   
2. Press e to edit and correct all metadata for this one book. Press OK to save changes. (I believe it is at this point that changes are commited to metadata.db)
   
   
3. Immediately press e again to relook at the metadata for this book. Is the metadata still correct?
   
   
4. If yes. Close down Calibre completely.
   
   
5. Reload Calibre and find the book again. Press e again and look at the metadata for a 3rd time. Is it still correct?

The above should tell you whether your changes are 'taking'.

P.S Do not do any Converting or Save-to-Disk.

[Starson17]

Originally Posted by Merischino

to "correct" the metadata for all of Anne McCaffrey's 105 works (in my database), i perform a number of different procedures, all of which include the steps of selecting the works(s) in a calibre GUI list, hitting "e" to open what I will call the 'metadata editing window" and then depending on whether there are multiple items or a single item selected, making changes to the Author and the Author Sort fields,

This all sounds correct. Nonetheless, what you think you entered into author_sort is not what I've found there. When multiple books are selected, it's a "bulk" metadata edit. When only one, it's single metadata edit. There's an option "Automatically set author sort" on the bulk edit screen - are you selecting that option? Can you perform your steps, then select a single record and pressing "E," check it to see if it sets the way you expect?

Once all that is done, I close out that metadata editing window, I hit "c" to "convert" the file back into the same format it was when I began.

This seems a bit odd - you don't need to do any format conversions when setting metadata?

and expect that my library would now sort according to the new information, irrespective of whether or not I have overwritten the existing files with the new data. E.g., I would expect "Power Lines" on the left under "Title" to be the same, "Power Lines" to the right under author to have changed, and to read "McCaffrey, Anne", and if I removed the search query and sorted the total library on "author" I would expect the book "Power Lines" to appear in the "M" section for McCaffrey, Anne and not in the "A" section, and not to show "Anne McCaffrey" in either the "a" or the "m" section, but to show "McCaffrey, Anne" in the "m" section of that list.

I agree. If it isn't correctly showing author or isn't correctly sorting, what do you see when checking a single book? Is author_sort right or wrong?

Now, I do not know when my changes to the metadata within the GUI get reflected in the Metadata.db.

It happens when you close the metadata edit screen.

Whether that happens instantaneously or only if/unless I convert (or re-convert, since it's going to the same format) or (ugh) "save to disk".

No, you don't have to Save to Disk.

But, only one thing can I say is true:
I have put a lot of time and effort into changing the metadata, by hitting "e" and editing the metadata in that window, to reflect what I have b een posting here as what I believed to be in my database.

Namely, that all my author fields should have the same exact entry as the author-sort field.

But they don't. We need to find out if author_sort ever had the right data.

And, again I thank you for your help and apologize for my frustration, which I know must be coming through. It's not directed at you.

I understand. And when someone here posts something that makes you think we're blaming you, that's not it. We just don't understand what's happening. If you will work through it, we'll track it down. If it's a bug, we'll find it and get it fixed. If not, that's OK, too.

[Adoby]

A quick sanity test. If this fails, then it could be some problem with rights or a corrupt install. Or that the files/folder are opened by some other process and cant be moved/renamed. Or something else ...

I just tried this on my library, it worked fine:

1. Highlight all Anne McCaffrey books. Every single one at the same time!
2. Right click on one of the highlighted books and select Edit metadata -> Edit metadata in bulk. (Or press 'e'.)
3. Enter "McAffrey, Anne" as Author.
4. Enter "McAffrey, Anne" as Author Sort.
5. Press OK.
6. Take a deep breath while Calibre renames the folders.
7. Exit and restart Calibre.
8. Sort on Author.

Now all Anne McCaffrey books should sort together, on M for McAffrey, and in the GUI the Author name displayed should be "McCaffrey, Anne".

Edit: Jackies test above is simpler. Try that first.

[Starson17]

Originally Posted by Merischino

I guess I'd like to ask what this tweak is. Or whether using it, or not using it, would affect the behavior of the sorting for those of us (like me!) who want the sorting and the appearance of the name in that sorted list to be the same.

Yes, one option in the tweak is to copy the name unchanged to author_sort.

eg Author: McCaffrey, Anne
Author-sort: McCaffrey, Anne
Appearance in sorted list looks like: McCaffrey, Anne

The "Appearance in sorted list" is always just what's in the author field.

Not sure whether this is even relevant given that my metadata.db file does not reflect my changes to my metadata, but assuming we can fix that and it does ultimately reflect my changes, would my entering the exactly same data into both of those two fields cause a conflict?

You are free to enter whatever you want in the two fields. They never conflict. One shows up in the screen, the other just controls sort order.

[speakingtohe]

Perhaps a stupid question but there is no chance that you are importing books of a DVD or CD and they are read only attribute?