Importing and maintaining title, author

[AndrewKantor]

I'm loving Calibre (kudos to your wife on picking a good name!). But something is driving me nuts.

I have a collection of e-books. I carefully went through them and standardized the file names: Lastname, Firstname - Title.epub (e.g., Pratchett, Terry - The Color of Magic.epub).

But when I import these into Calibre, it seems to come up with its own naming scheme. The author is usually changed to Firstname Lastname and the titles lose whatever capitalization corrections I made (e.g., "The Color of Magic" becomes "The color of magic").

It seems that Calibre is pulling the data from the file itself. But is there any way to either force it to use my structure, or at least tell it to use "Lastname, Firstname" for authors?

Thanks for any help you can give!

[kovidgoyal]

If you want metadata to be read from only the file names (as opposed to inside the file) there is an potion to set in Preferences->Add/Save->Adding books

[Starson17]

Originally Posted by AndrewKantor

I'm loving Calibre (kudos to your wife on picking a good name!). But something is driving me nuts.

I have a collection of e-books. I carefully went through them and standardized the file names: Lastname, Firstname - Title.epub (e.g., Pratchett, Terry - The Color of Magic.epub).

That was my naming scheme, too. I've just finished importing my collection into Calibre. Since your names are good, just set your regex importing expression and tell Calibre to get the author name and title from the filename.

But when I import these into Calibre, it seems to come up with its own naming scheme.

Calibre doesn't actually have a naming scheme of it's own. It knows the author name and the title of the book. From those (and from other optional metadata, like the series name and number), you can create any naming scheme you want. Your naming scheme is used when you export your books.

the titles lose whatever capitalization corrections I made (e.g., "The Color of Magic" becomes "The color of magic").

I used to see that when I fetched metadata. Fetched metadata often has bad capitalization. There is an option to prevent the author/title from being overwrittten during metadata fetch.

It seems that Calibre is pulling the data from the file itself.

It can also come from bad metadata inside the book.

But is there any way to either force it to use my structure, or at least tell it to use "Lastname, Firstname" for authors?

You can't directly change Calibre's storage structure. You shouldn't need to. If you insist, you can lie to Calibre and tell it that the author's firstname is the lastname. There is a special tweak that will assist you in doing this, while keeping the sorting structure correct (by lastname) but I haven't tried it.

You might want to read the FAQ, particularly the section entitled:
Why doesn’t calibre let me store books in my own directory structure?¶

[AndrewKantor]

Ah, wonderful! Thank you both. Regex has always given me fits, but I was able to figure enough to get it to import properly. W00t!

[Starson17]

Originally Posted by AndrewKantor

Ah, wonderful! Thank you both. Regex has always given me fits, but I was able to figure enough to get it to import properly. W00t!

here is the regex I'm currently using, it should handle the naming scheme you use for your books, and several others, particularly, series names and numbers:

^((?P<author>([^\-_0-9]+)(?=\s*-\s*)(?!\s*-\s*[0-9.]+)|\b))(\s*-\s*)?((?P<series>[^0-9\-]+) (- )?(?P<series_index>[0-9.]+)\s*-\s*)?(?P<title>.+)

If you haven't already figured it out, I'd recommend you turn on the option to get metadata only from filenames and turn off the option to overwrite author/title metadata during fetch. You can then highlight multiple files and let Calibre go get covers and other"social" metadata, if you want.
If you have problems during the import, let us know.

[rollercoaster]

@Starson17
thanks for the regex.. this is great..

can it be modified to ignore brackets around series like -

First last - (Series 1) - title.ext
First last - [Series 1] - title.ext

and i dont know if it is possible but here it goes, can we use regex such that if author name has a coma it swaps first/last name. like -

First Last - title.ext -> Author name = "First Last"
Last, First - title.ext -> Author name = "First Last" instead of "Last, First"
Last, First M.- title.ext -> Author name = "First M. Last" instead of "Last, First M."

[Starson17]

Originally Posted by rollercoaster

@Starson17
thanks for the regex.. this is great..

can it be modified to ignore brackets around series like -

First last - (Series 1) - title.ext
First last - [Series 1] - title.ext

Yes. [\(\[]? means optional open parentheses or square brackets. Try this: (untested by me)

^((?P<author>([^\-_0-9]+)(?=\s*-\s*)(?!\s*-\s*[0-9.]+)|\b))(\s*-\s*)?([\(\[]?(?P<series>[^0-9\-]+) (- )?(?P<series_index>[0-9.]+)[\)\]]?\s*-\s*)?(?P<title>.+)

and i dont know if it is possible but here it goes, can we use regex such that if author name has a coma it swaps first/last name. like -

First Last - title.ext -> Author name = "First Last"
Last, First - title.ext -> Author name = "First Last" instead of "Last, First"
Last, First M.- title.ext -> Author name = "First M. Last" instead of "Last, First M."

You can't use regex to do this - it has to be done in the code. I've considered writing code to do this (you''ll see a discussion of this in some of my posts), but haven't done it. Some author names have commas (Jr, the IIIrd) so that code could cause problems. I wrote code to swap at the comma to deal with authors who have middle names or middle initials, but I decided this autoswap would complicate things too much. (I'm really, really trying not to ask Kovid to add any more options. He's already added 3 for me. That's enough.)

[chaley]

Originally Posted by AndrewKantor

But is there any way to either force it to use my structure, or at least tell it to use "Lastname, Firstname" for authors?

If you are happy with calibre's default 'firstname lastname' author ordering, don't read on.

If you want to use "Lastname, Firstname" for authors, then you probably also want to change how calibre copies the author names to the author_sort field. If you don't, then you must manually edit author_sort every time you import a new book or change an author's name to avoid sorting by author displaying the list in a very strange order.

You can control how the copy using a calibre 'tweak' that changes how author_sort is generated from author. In the file tweaks.py, change author_sort_copy_method to either copy or comma (see below for more detail). Tweaks.py is found in your settings directory, which for me on windows 7 is C:\Users\charles\AppData\Roaming\calibre.

From tweaks.py:

# The algorithm used to copy author to author_sort
# Possible values are:
#  invert: use "fn ln" -> "ln, fn" (the original algorithm)
#  copy  : copy author to author_sort without modification
#  comma : use 'copy' if there is a ',' in the name, otherwise use 'invert'
author_sort_copy_method = 'invert'

I use 'copy'.

If the current author_sort fields are not correct, then once you set the tweak you can reset all the author_sort fields in one swoop. First, verify that the author names are all as you want them. Then select the books you want to change, press E to get the bulk metadata editor, check the box "automatically set author sort", and push OK. The author_sort field for all the selected books will be recomputed from the author field according to the value of the tweak.

[rollercoaster]

@Starson17 that is amazing. thank you thank you

I am happy even if we cant switch First/last as that gets corrected when we lookup for metadata.

[Sabardeyn]

chaley,

I've seen references to this discussion several times. Maybe I've been dense every time I've looked, but I'm not sure I understand what you're saying here.

Does this make a purely cosmetic change to Calibre, effectively changing how author names are displayed in the Author(s) column of the main window? Or, are your changes altering calibre's file structure instead? Or both?

In other words, I have "Stephen King - The Stand.epub" (for example).

Current Calibre functionality:
Author = Stephen King (as shown on Edit Metadata screen)
Author Sort = King, Stephen (ditto)
File Structure results in the path: .../Stephen King/The Stand (nnn)/
Author Column = Stephen King, listing it in the K's.

And after using the technique that you show above, it would alter the functionality to which of the following?
1) File Structure results in the path: .../King, Stephen/The Stand (nnn)/
2) Author Column = King, Stephen listing it in the K's.
3) Both of the above changes

Please explain. Because I might have to contact someone regarding nominations for sainthood - at least!

[chaley]

Originally Posted by Sabardeyn

Does this make a purely cosmetic change to Calibre, effectively changing how author names are displayed in the Author(s) column of the main window? Or, are your changes altering calibre's file structure instead? Or both?

Both, because calibre uses the authors when building the path.

And after using the technique that you show above, it would alter the functionality to which of the following?
1) File Structure results in the path: .../King, Stephen/The Stand (nnn)/
2) Author Column = King, Stephen listing it in the K's.
3) Both of the above changes

3 -- both. Calibre believes that whatever is put into author is really the author, and uses that name is several places. You have identified 2 of them, but there are also some uses when managing devices (file structure, metadata).

For example, I enter (and see) David Weber as "Weber, David". Because author_sort is set to author, he appears correctly in the 'W' section. In calibre's library, one of his books is stored as (converting to forward /s) "/Library/Weber_ David/1633 (49)/1633 - Weber_ David.epub". On my Sony Pocket, the file name is "/database/media/books/Weber, David/1633 - Weber, David.epub"

Please explain.

Did I?

Because I might have to contact someone regarding nominations for sainthood - at least!

Are you talking about me?!? Nahhh

[Starson17]

Does this make a purely cosmetic change to Calibre, effectively changing how author names are displayed in the Author(s) column of the main window? Or, are your changes altering calibre's file structure instead? Or both?

Originally Posted by chaley

Both, because calibre uses the authors when building the path.

Or to look at it differently, it does neither. There are no changes made to either display or storage. He merely tells Calibre that the "author" is lastname, firstname, when other users tell Calibre that the author's name is "firstname lastname." Whatever you tell Calibre about the author's name will be used as that name.

Anyone can do this without needing chaley's mods (and in fact, IIRC, that's what he did before he wrote his code.) His code just makes it easier to control the parts of Calibre that automatically reverse "firstname lastname" to "lastname, firstname" when creating the author_sort field.

[Sabardeyn]

Chaley,

You sir, should indeed be sainted! I will have to look into this further because I will definitely make use of this. Honestly, I don't think many other users understand what you've done.

Calibre's current implementation of authors and titles is something I find less than ideal. Because what I've been taught to use, and how it is implemented, are not the same.


Starson17,

While that might be accurate, if I read that answer first I would only be further confused.

[Starson17]

Originally Posted by Sabardeyn

Calibre's current implementation of authors and titles is something I find less than ideal. Because what I've been taught to use, and how it is implemented, are not the same.

What do you mean by "what I've been taught to use?"

Starson17,

While that might be accurate, if I read that answer first I would only be further confused.

Nonetheless, the way I described it is how the code really works, and that's how I think of it. Chaley didn't change anything about the author, the display of the author or the author_sort field. Those details are integrated too deeply into the design of calibre to be changed. He just changed how calibre copies the author into author_sort - which makes it easier to use caliber with the ln, fn format.

For me it's easier to think about how it really works.

[Sabardeyn]

Starson17

I was taught in the USA. So the library files books under author's LN, FN. Titles beginning with an article of speech are "re-named" for filing purposes, by appending the article of speech to the end of the title. As such, calibre only has a partial implementation, at least from my biased viewpoint, because these things are done, but not implemented, if you understand me.

However, calibre has been around longer than I have been dealing with ebooks. It's also used internationally. Perhaps a previous discussion determined that this method was best - all things considered. I have no way of knowing about that. I just find its partial implementation confusing.

As to the coding, I figured that you were being "programmatically correct", but despite that, your answer was not clear. Not to me anyway. But then neither was chaley's comment (message #8). Like I said, I can be a bit dense at times. But the implication of what chaley was offering was not clear to me until after he answered my questions (message #10).