Saving to disk with "Unknown" in author field

[Joe42]

Calibre can't handle having "Unknown" in the author field when saving to disk despite my save template working perfectly for anything else.

Here's my save template:

{author_sort[0]}/{author}/{series:||} {series_index:0>2s|| -} {title}

When I save to disk, my ebooks are saved with a simple folder & file name structure. Going by the first letter of the author's last name, a folder is created and named with that letter, then inside that folder another folder with the author's name is made, and inside that folder is where the file goes, and it get's named with just the title of the book.

Example:

A
   Will Adams
      The Alexander Cypher.epub
   Mark Ames
       Going Postal.epub

B
   Alan Ball
       American Beauty.epub

You get the idea.

Anyway, Calibre should just create a folder called "U" and inside that one called "Unknown" then drop the ebooks in there, but instead the python scripting breaks down and I get the following error, no folder structure for "U" is made and no file is saved:

	Traceback (most recent call last):
	  File "site-packages\calibre\library\save_to_disk.py", line 437, in save_serialized_to_disk
	  File "site-packages\calibre\library\save_to_disk.py", line 305, in do_save_book_to_disk
	ValueError: Failed to calculate path for save to disk. Template: {author_sort[0]}/{authors}/{series:||} {series_index:0>2s|| -} {title}
	Error: string index out of range

This error has been a constant pain in my backside since I started using Calibre somewhere in the version 6.xx area and it still exists with the latest version, 0.8.25.

Any ideas?

[DoctorOhh]

Originally Posted by Joe42

This error has been a constant pain in my backside since I started using Calibre somewhere in the version 6.xx area and it still exists with the latest version, 0.8.25.

Any ideas?

Submit a bug report.

Kovid is good, but I don't believe he has ever claimed to be Psychic.

[theducks]

OMG it is amazing that I have not tripped that bug.

I use a similar folder structure (Authors_A)

Authors_{author_sort[0]}/{author}/

As a work around:
Change all Unknown to (the word) Unknown (or Authors Unknown)
(the first is probably a blank value that only displays as Unknown)

[Joe42]

Originally Posted by dwanthny

Submit a bug report.

Kovid is good, but I don't believe he has ever claimed to be Psychic.

I'll do that. Thanks.

Originally Posted by theducks

OMG it is amazing that I have not tripped that bug.

I use a similar folder structure (Authors_A)

Authors_{author_sort[0]}/{author}/

As a work around:
Change all Unknown to (the word) Unknown (or Authors Unknown)
(the first is probably a blank value that only displays as Unknown)

Typing Unknown in the author field doesn't work, same result. Hadn't thought of using Authors Unknown. I'll give that a try. Thanks.

I forgot to add that if I browse in Win Explorer to my library, those with unknown author are actually named as such and are placed in a folder called "Unknown". Here's a screenshot of just one example. I have several separate libraries with unknown authors.


*Using "Unknown Authors" works fine.

[Starson17]

Originally Posted by theducks

Change all Unknown to (the word) Unknown (or Authors Unknown)
(the first is probably a blank value that only displays as Unknown)

"Unknown" as an author has entries in the metadata.db database for both author and author_sort. In fact, IIRC, there is an author table, with an author name field and a sort field for the author names, as well as a book table, with an author_sort field. All 3 of those author fields are populated with "Unknown" on an English system and with the corresponding translation for other languages.

"Unknown" is handled specially on input, but I'm surprised at this error on output.

[chaley]

The problem is that author_sort is empty, which is arguably appropriate when there is no author. You can see this by sorting on author -- the "Unknown"s will show up at the beginning of the list unless author_sort has been manually set. Note that the same thing will happen if you clear author_sort by hand.

To avoid this problem, set the author sort for all books where it is empty. Search for author_sort:false, select the books, open bulk metadata edit, check the "Automatically set author sort", and press OK. Alternatively, set it to what you want, which may be something other than 'Unknown'.

[Starson17]

Originally Posted by chaley

The problem is that author_sort is empty, which is arguably appropriate when there is no author. You can see this by sorting on author -- the "Unknown"s will show up at the beginning of the list unless author_sort has been manually set. Note that the same thing will happen if you clear author_sort by hand.

I wonder if this has changed? My db has author_sort set to Unknown for all my Unknowns? I did a quick test by adding an empty book, and author_sort was set to Unknown for it. Under what circumstances is author_sort left empty? Is it during import, with an empty author, or only by clearing the author_sort field?

[chaley]

Originally Posted by Starson17

I wonder if this has changed? My db has author_sort set to Unknown for all my Unknowns? I did a quick test by adding an empty book, and author_sort was set to Unknown for it. Under what circumstances is author_sort left empty? Is it during import, with an empty author, or only by clearing the author_sort field?

I also haven't been able to make it happen. I have tried empty books, books with empty file-as in the OPF, etc.

My suspicion is that the OP has books from way-back-when when we might have handled null values incorrectly. Alternatively, author_sort was explicitly set to empty at some point. Pure speculation on my part.

[Starson17]

Originally Posted by chaley

My suspicion is that the OP has books from way-back-when when we might have handled null values incorrectly.

Yes, there was a period it was handled incorrectly that I recall. It also explains why others haven't seen this and why it's bugged him for a long time. I suspect a quick search and correction on the nulls is all that's needed. It may already be fixed if he's changed to Unknown_Author. That would have populated his nulls and he won't get them back without some intentional effort.

[Starson17]

BTW, "Unknown" as an author name (or it's set-language equivalent) is handled specially for various options, such as Merge and Copy to Library. If the database holds "Unknown" (or language equivalent), I treated it as an empty field in a Merge, etc. so that a book having an author name of Unknown in the first selected record is overwritten by the author name of the second selected book if it's other than "Unknown." If Automerge is set, the same thing happens with Copy To Library.

Using "Unknown Author" instead of "Unknown" will be treated as a real author name, not a null for those operations. If you simply don't know the author, I'd use "Unknown," but if no one knows the name, "Unknown Author" might be a better choice so you aren't tempted to go looking for what can't be found.

[Joe42]

I don't understand what you guys are saying. It's obviously a bug in Calibre, hence the error msg. Kovid killed my bug report saying it's invalid and provided no reason other than "see thread for details." Too bad there's no details to see.

I guess I'll just live with using "Author Unknown" since that seems to work fine. Is it possible for me to change the default "Unknown" to "Author Unknown"?

As for my ebooks possibly being old, I've been building my collection for less than 1 year, so none of the files are very old. Besides, the files themselves should have no bearing on how Calibre exports them since all it's doing it creating folders and naming files according to my template.

Also when exporting, Calibre automatically updates the metadata within the files themselves, so older formats shouldn't be a problem.

I did find out that when author is Unknown, whether it's automatic or I type it, that only applies to author_sort, not author.

[Starson17]

Originally Posted by Joe42

I don't understand what you guys are saying. It's obviously a bug in Calibre, hence the error msg.

It's not a bug now. It may have been in the past.
Your problem is that you had some null author_sort records. It shouldn't be possible to have that unless you specifically nulled them. It could have been by mistake or an old bug.

Try this:

1) First search for the problems:

author_sort:false

If you get results, check that the author is "Unknown" for all.

If it is, select them all, enter bulk metadata edit by pressing "e" and set both the author and author_sort to "Unknown." (if it isn't, then you've got some null author_sort records for valid author names, and you should fix those separately) That should solve your problem.

I would also search for author:"Author Unknown" and change that and the author_sort back to Unknown for the reasons above.

[GoingDown]

I tested this in Linux version in Calibre, and it seems to give the error message in OP if the "Unknown" is typed so that first letter is uppercase.

If the whole string is lowercase, then saving works, but if I change it back to capital U, then it breaks again. Note that I am talking about Author - field, not author_sort.

Strange

Edit. Tested again by editing one book and renamed author to Unknown. Saving to disk failed. Renamed author to "unknown" and it worked. So it seems to be bug

[chaley]

The error reported by the OP regards author_sort. The contents of Author is not relevant. Specifically, the error message says that author_sort is empty, meaning author_sort[0] is undefined, which calibre is reporting. This is correct behavior -- calibre is reporting the error that the template asks for information that is not available. There is no "obvious bug" here.

The next question is "how did author_sort become empty?" There are several ways to do this, including the "manage authors" dialog and metadata edit. Apparently it was possible to do this during import sometime in the past, but I think not since middle 2010. Starson17 and I have tried several ways to create a book with an empty author_sort and have not been able to do it.

If someone has a way to enter a new author or import a new book and have the author_sort remain empty, I would like to see it. The author name must not exist before the book is imported/changed. If the author already exists, then the test is flawed because that author might already have a stored empty author_sort value.

@goingdown: you don't say which version of calibre you are using. It would be useful to know whether or not the authors Unknown and unknown both simultaneously exist and if so whether the author_sort values for these two authors are defined, something you can see using "manage authors" (right-click on an author in the tag browser and choose "manage authors"). It also would be useful to know what the author_sort value is for a failing book after you make the change, and whether calibre considers that value as correct (not pink) -- both of these can be seen using edit metadata.

[Starson17]

Originally Posted by chaley

Starson17 and I have tried several ways to create a book with an empty author_sort and have not been able to do it.

If someone has a way to enter a new author or import a new book and have the author_sort remain empty, I would like to see it.

I would like to emphasize Charles' comments. I recall looking closely at the code for adding a book to the calibre library. If calibre can't find an author, it inserts the local language word for "Unknown" as the author. The author name, whatever it is, is converted to an author_sort name. For a long time, author_sort was relatively inaccessible. It no longer is inaccessible. You are perfectly free to force it to be null, and if you do, you will get the error of the OP.

Neither Charles nor I can find a way to accidentally make it null during an import (except as below). There may be one (other than below), and if there is, that's a bug.

I'll also comment that author_sort is defined for a book, not an author. There is a corresponding field for the author called simply "sort." You can see "author_sort" in a book record. You can see the "sort" field for the author in Manage Authors. The sort field in the author table is used for the author_sort field in the book table if the author was previously defined.

If the OP has accidentally nulled the sort field in the author's table for the "Unknown" author (using Manage Authors), it will produce a null author_sort field for each newly added Unknown author. This needs to be fixed with Manage Authors and all old books with this problem also need fixing to solve the OP's problem. I just tested it to confirm this. Nulling the sort field for the author "Unknown" produces null author_sort fields for new books with unknown authors.

This problem applies to all authors, not just the special Unknown, but it's more likely that someone tried to blank out the Unknown author than any other. Generally, calibre will catch any attempt to null the author field and replace it with Unknown, but it's possible that there is some situation where calibre caught the attempt to blank the author field (which is prohibited due to the filename requirements of the library), but didn't catch the blank "author_sort" or "sort" fields, which are both permitted.