Vast rewrites when importing a wrongly cased author

[kacir]

Originally Posted by kovidgoyal

Renaming files is pretty fast, it shouldn't really bottleneck things (it works by hard linking and then deleting the originals, in a transactional manner). And as theducks said, updating the database is of no consequence, since it only needs to update a single row for a case change.

But it also updates all the opf files.
And when you are on an SSD, you only watch helplessly as your disk blocks are getting rewritten again and again and AGAIN when you correct the author name back.

One memory cell on an SSD with MLC can only be rewritten 7000 times before it fails. This is a very large number, especially with all wear leveling routines build into each modern SSD.
For a typical consumer-level SSD nowadays, with TLC nand memory cells the number of rewrites is significantly lower, and is amplified by the fact that you can not delete (rewrite) an individual memory cell, you have to delete the entire block of cells.

It is difficult to know precise numbers, because they are not advertised by manufacturers. Some manufacturers (Kingston, I think) will not even tell you what kind of memory cell is in your SSD.

So, every time I see Calibre to import a book with differently capitalized author name I cringe. It might mean that quite a lot of files have to be renamed and many opf files written from the scratch. I know that a file or a directory name rewrite is a change of a few bytes, but the entire block of NAND memory cells on my SSD will have to be zeroed by a TRIM for a smallest change. And the individual opf files and file names are usually scattered all over the place and not in a small consecutive block.

[kovidgoyal]

Stop worrying about your SSD. It can take it. SSDs have plenty of spare blocks to deal with precisely this issue. I've been using calibre on a single SSD for almost six years now, with no issues. Indeed, your system log files alone will generate far more write activity than calibre renaming authors. And I wont even mention web browser caches, which are probably at leasta couple of orders of magnitude more write heavy.

[kovidgoyal]

Also, I cannot reproduce the original issue in the first place (in calibre 3.15). Steps I tried:

1) Import a book
2) Edit the author of the book, changing it to all uppercase
3) Import the book again
4) Click on add duplicate
5) Both book entries have upper case authors

[sciurius]

Looking at the inodes of the files before/after the change, the inodes of epub and cover do not change so the hard linking works.

[kacir]

Originally Posted by kovidgoyal

Also, I cannot reproduce the original issue in the first place (in calibre 3.15). Steps I tried:

1) Import a book
2) Edit the author of the book, changing it to all uppercase
3) Import the book again
4) Click on add duplicate
5) Both book entries have upper case authors

You are right.
The problem has been already solved.
Calibre 3.16 no longer changes author case when importing a book with author with a different case (Charles DICKENS vs Charles Dickens).
You can import reading Metadata from e-book file or from filename.

Thank you Kovid

EDIT:
Obviously I do not worry about the health of my [bloody expensive] large SSD that much(*), when I did not notice when Calibre ceased to change the author case during import.

(*) not as much as at the beginning, anyway ;-)
I do keep an eye on usage

[kovidgoyal]

You are welcome, though I did not do anything

[sciurius]

I upgraded to 3.16 and I still get the vast updates when importing a book with author with a different case.

To be more precise:

- I have 100s of books with author "Auteurs, Diverse" and author_sort "Auteurs, Diverse". (I keep author the same as author_sort).

- I import some books that have author "Diverse auteurs" and (derived) author_sort "auteurs, Diverse"

- For these books I copy author_sort into author using Metadata > Search & Replace.

Several minutes later all "Auteurs, Diverse" have been changed to "auteurs, Diverse"

[kovidgoyal]

That's not an author with a different case, that's an author with a different value. And if you use bulk metadata replace then you will get rewrites for all the book you have done the replace on.

[sciurius]

I get rewrites for the books that I operate on (of course) -- but the 100s of other books that did have "Auteurs, Diverse" are also changed.

[sciurius]

To summarize:

Case one: When I select one book and change the author to something else, that one book is affected. This is good.

Case two: When I select one book and change the author only in case (e.g. "unknown" to "Unknown"), all books currently having that value as author are affected.

IMHO the latter behaviour is undesired. If I would want to change all authors, I can select all and replace all. But I have no way to avoid the mass affecting behaviour - in particilar if it happens during bulk import.

I would propose that, if an author is changed in case only, the change is refused.

[kovidgoyal]

If you deliberately change the case of an author, all the books by that author are affected, that is the correct and desired behavior. And no, during import no author case is changed, as I demonstrated several posts before with mmy set of test steps.

[sciurius]

I can not reproduce the case changing during import with 3.16 so this may have been fixed.

Since it is part of my workflow to copy author_sort into author, a case change caused by copying will affect all books with the same author. If this is as it should be I can live with that.

[kacir]

Originally Posted by sciurius

Case two: When I select one book and change the author only in case (e.g. "unknown" to "Unknown"), all books currently having that value as author are affected.

IMHO the latter behaviour is undesired.

I personally think that this behavior is exactly as it is supposed to be. You make a change to the database, it propagates to all the records. I do not think it is desirable to have Unknown AND unknown authors in the authors table in database. And you can only assign a value from authors table as an author for the book, exactly as design of a relational database dictates.

[sciurius]

You make a change to the database, it propagates to all the records.

Yes. This wouldn't be a big deal if it were only a database update. But in the case of changing an author name all associated files and folder on disk are changed, which may take a long time if there are many.

Originally Posted by kacir

I do not think it is desirable to have Unknown AND unknown authors in the authors table in database.

Sure, that's why I suggested to refuse the change.

[ilovejedd]

Originally Posted by kacir

EDIT:
Obviously I do not worry about the health of my [bloody expensive] large SSD that much(*), when I did not notice when Calibre ceased to change the author case during import.

(*) not as much as at the beginning, anyway ;-)
I do keep an eye on usage

I've switched to SSDs (120GB - 1TB) for all OS drives on my desktops and laptops. The oldest still in use, an Intel X25-M G2 120GB (iirc, 5000 P/E cycles so good for around at least 600TB NAND writes) is still at 100% health after almost 7 years of usage. The sole SSD I have that's below 90% health is a 128GB Crucial m4 that's also used as TV buffer in the HTPC (~100+GB writes per day).

Kovid's right. Browser cache does far more damage than Calibre ever will (and despite that, I definitely still want cache, tmp, pagefile, etc on SSD instead of HDD). Besides, wear leveling techniques are pretty advanced anyhow. The worst case write amplification I've seen thus far with my usage (which is probably worse than average) is around 3x.