Change Author - Version 0.9.4

[BetterRed]

If I change the title or author of a book, which has a book file open in an editor then I get a message



If I click OK, the metadata is changed but the book folder remains as it is. Now I have a book whose Title and Author in the Database don't match with the Author and Book folder names or the book files

Imagine I have "Bring Out The Bodies" by "Hillary Mantell" as a PDF. I open the PDF and see that its actually "Bring Up The Bodies" by "Hilary Mantel" so I immediately change the Title and Author and get the message above. If I click OK then I will be able to read some more of the book, and attach a couple of stickies and save it, it will be written to

Hillary Mantell/Bring Out The Bodies/Bring Out The Bodies - Hillary Mantell.pdf

Now if I save the PDF to the desktop I get a file named "Bring Up The Bodies - Hilary Mantel.pdf" - if I copy that version back to calibre I will have two files in the Hillary Mantell/Bring Out The Bodies folder

Bring Out The Bodies - Hillary Mantell.pdf
Bring Up The Bodies - Hilary Mantel.pdf

On a database check (second step) the first file will be flagged as an extra format - which I can delete. But I still have the anomaly of the folder names Hillary Mantell/Bring Out The Bodies.

It's all very confusing

I suggest that the error message have a Cancel option, which would be the default, and if its chosen then the Calibre Title and Author remain as is and conformant with the existing folder and book file names.

Then the user can close the book file, then correct the title and author, so that the Calibre database, the folder names, and file names are all in accord with one another.

BR

[opitzs]

Do you use dropbox, or something similar?

[drjenkins]

I am having the same problem with 0.9.4. If I revert back to 0.9.3 the problem does not occur. The problem occurs if I change either the Author or Title. For example, as a test I changed the name of "The $100 Startup" to "The 100 Dollar Startup". I get the following error:

calibre, version 0.9.4
ERROR: Unhandled exception: <b>WindowsError</b>:[Error 2] The system cannot find the file specified: u'U:\\My eBooks\\CalibreLibraries\\EvaluationLibrary\\Chris Guillebeau\\The 100 Dollar Startup (2362)\\The 100 Dollar Startup - Chris Guillebeau.epub'

Traceback (most recent call last):
File "site-packages\calibre\gui2\metadata\single.py", line 466, in accept
File "site-packages\calibre\gui2\metadata\single.py", line 440, in apply_changes
File "site-packages\calibre\gui2\metadata\basic_widgets.py", line 96, in commit
File "site-packages\calibre\library\database2.py", line 2537, in set_title
File "site-packages\calibre\library\database2.py", line 665, in set_path
File "site-packages\calibre\library\database2.py", line 1520, in add_format
File "genericpath.py", line 49, in getsize
WindowsError: [Error 2] The system cannot find the file specified: u'U:\\My eBooks\\CalibreLibraries\\EvaluationLibrary\\Chris Guillebeau\\The 100 Dollar Startup (2362)\\The 100 Dollar Startup - Chris Guillebeau.epub'

After clicking "OK" I get:

calibre, version 0.9.4
ERROR: Permission denied: Could not open file. Is it being used by another program?

Traceback (most recent call last):
File "site-packages\calibre\gui2\metadata\basic_widgets.py", line 96, in commit
File "site-packages\calibre\library\database2.py", line 2537, in set_title
File "site-packages\calibre\library\database2.py", line 665, in set_path
File "site-packages\calibre\library\database2.py", line 1519, in add_format
File "site-packages\calibre\library\database2.py", line 1377, in copy_format_to
File "site-packages\calibre\utils\filenames.py", line 308, in copy_path_to
File "site-packages\calibre\startup.py", line 156, in local_open
OSError: [Errno 13] Permission denied

The folder is still called
Chris Guillebeau\The $100 Startup (2362)
and the original files
The $100 Startup - Chris Guillebeau.epub
The $100 Startup - Chris Guillebeau.mobi
are joined by the new file
The 100 Dollar Startup - Chris Guillebeau.epub

If I repair the library, the new file is deleted.

When I revert to 0.9.3 and repeat the above procedure there are no erros and the end result is that the folder and files are renamed:
Chris Guillebeau\The 100 Dollar Startup (2362)
The 100 Dollar Startup - Chris Guillebeau.epub
The 100 Dollar Startup - Chris Guillebeau.mobi
and the library does not need repair.

For now I'm sticking with 0.9.3. FYI, I'm not using Dropbox or anything similar.

[drjenkins]

On second thought, maybe I do use "something like Dropbox". My U:\ drive is mapped to a network share on my LAN. If I repeat the above test on a library located on a local hard drive 0.9.4 works as expected. At least in my case it appears that 0.9.4 has an issue with mapped drives.

[kovidgoyal]

Originally Posted by drjenkins

On second thought, maybe I do use "something like Dropbox". My U:\ drive is mapped to a network share on my LAN. If I repeat the above test on a library located on a local hard drive 0.9.4 works as expected. At least in my case it appears that 0.9.4 has an issue with mapped drives.

0.9.4 uses hard links when changing title/author, for performance, apparently hard links dont work with whatever network filesystem you're using. Even worse, the windows CreateHardLink function appears to be failing silently. Sigh. I'll add an explicit check to ensure the hardlink was created for the next release, which should take care of the problem, but generally speaking, you should not put your calibre library on a network share, network filesystems are broken in lots of different ways.

[drjenkins]

Thanks Kovid. I've temporarily moved my live libraries to a local hard drive, using the network share as a backup. I've never had a problem with Calibre and network shares before and it's convenient to reach the libraries from any workstation on the Windows LAN.

[BetterRed]

My Calibre Library is on an internal hard disk, I don't use Dropbox or anything like it, I run Calibre on a standalone system. I dont have any SSD's, or RAM Disks and the databases are within the relevant library folders and there are no hard, soft or alias links to them - i.e a vanilla, white bread installation.

The drive where I have the Calibre libraries has thousands of hardlinks (unrelated to Calibre - related to analysis of telemetry data). Some files have 1023 (the maximum) hardlinks so that I've had to create second instances of the data. I have some NTFS drives that I mount as needed in a disk dock, two of those disks also have lots of hardlinks - some of which were created under NT3.51 in 1998. They all work.

Since Vista (maybe XP, can't remember) every Windows system has Windows\winsxs folder, 90% of the files in there are hardlinks to file elsewhere, I've seen files with up to 6 links in there. And there are over 11,000 subfolders in my winsxs folder.

Even lowly notepad.exe has four hardlinks

C:\Windows\notepad.exe
C:\Windows\winsxs\amd64_microsoft-windows-notepadwin_31bf3856ad364e35_6.1.7600.16385_none_9ebebe8614be1470\notepad.exe
C:\Windows\winsxs\amd64_microsoft-windows-notepad_31bf3856ad364e35_6.1.7600.16385_none_cb0f7f2289b0c21a\notepad.exe
C:\Windows\System32\notepad.exe

I've attached an image of a fragment of my Windows/System32 directory (which is where much of Windows lives), the little red arrows you see on the most of the icons indicates that the file has as least one other hardlink somewhere.

If you copy a file in Explorer from one folder to another on the same drive, then by default Explorer will Move the file by creating a hardlink at the new location and trashing the hardlink in the old location. You have to go out of your way to actually copy the file - ie do a cut & paste or a right click drag drop and select Copy. If Windows hardlinks didn't work then nor would Windows, so to suggest hardlinks may not work on Windows is a really big call. Some programs I wrote for NT3.51 manipulate hardlinks. They are working today, and the modify date on the relevant .cpp hasn't changed for over 12 years.

I very much doubt hardlinks have anything to do with the problem I originally described. I just changed the title on 521 books - worked perfectly - and fast, 10-20 secs at a guess. How much faster than before - dunno, never did it before.

If I try to copy a file into a book that already has a file of that type, and the existing file is open in an editor then I get a Permission Denied error message, the old file remains in the book, and any temporary files created by the editor remain where they are - in other words the 'request to replace the file' is rejected, quite rightly so

IMO that's what should happen if I try to change the title or author of a book that has one of its files open for editing. The Calibre database should keep the existing title and author, and Calibre should not delete temporary files the editing program created - which it does. And I don't believe I should end up with book file names that don't match the folder name and/or the Calibre title and author names.

If I created such a situation because of renaming things in Explorer and complained about something not working - then theducks and Dr NoNo etc would, quite rightly, be very with me.

BR

[BetterRed]

I wanted to let folks know that Calibre Version 0.9.5 has addressed my original problem.

To the extent that it can, Calibre will prevent the user changing a book's title or author if one of the book's files is opened for editing via this message "Could not change the on disk location of this book. Is it open in another program?"

The message was issued in these scenarios

RTF in Word
PDF in Calibre Exchange
cover.jpg in Irfanview

The message was not issued in these scenarios

TXT open in Notepad, Notepad++, Wordpad or VS Editor, open in Word is OK
EPUB open in Sigil

Calibre can only issue the message if the editor program locks the file. So if an editor doesn't cause Calibre to issue the message when you think it should, then almost certainly that will be because the editor program is not holding a lock on the file - Calibre cant fix that.

I have raised an issue on Sigil regarding EPUB files https://code.google.com/p/sigil/issues/detail?id=1741

I don't intend doing anything about TXT files.

Thanks BR

[Kiradel]

Originally Posted by drjenkins

I've never had a problem with Calibre and network shares before and it's convenient to reach the libraries from any workstation on the Windows LAN.

I am having the exact same problem with the latest version and second this sentiment. It's been working fine, as far as I can tell, and now I can't edit authors/titles at all. If I repeatedly hit Next it will eventually succeed, but any bulk operations are effectively killed at the moment.

If there was a client/server setup for Calibre that would be fine, but I store my library on my Windows Home Server for backup/redundancy purposes, and do all of my editing in Calibre from my desktop/laptop. I'd very much appreciate a fix for this ASAP.

Cheers, and thanks for a great program!

[DoctorOhh]

Originally Posted by Kiradel

I store my library on my Windows Home Server for backup/redundancy purposes, and do all of my editing in Calibre from my desktop/laptop. I'd very much appreciate a fix for this ASAP.

I can understand wanting to go back to whatever changed, but if you're updating/editing the library on your server then it isn't a backup. Many folks have their library on the local drive then use something like richcopy or synctoy to keep their server version current.

BTW a client / server version of calibre is in the works, no ETA at this time.

[Kiradel]

Originally Posted by DoctorOhh

I can understand wanting to go back to whatever changed, but if you're updating/editing the library on your server then it isn't a backup. Many folks have their library on the local drive then use something like richcopy or synctoy to keep their server version current.

BTW a client / server version of calibre is in the works, no ETA at this time.

To be clear, I meant that my Home Server already has RAID/redundancy/backups configured for all of the data I store on there, which means I don't need to worry about any of that any of the clients. Sure, there are other solutions, but it was just so simple and smooth as it was.

A client/server version would be a better solution, so I look forward to that when it's available.

For the moment, I'm just postponing any author/title cleanup work I was going to do, hoping the current issue gets resolved without me having to relocate my data.

[DoctorOhh]

Originally Posted by Kiradel

To be clear, I meant that my Home Server already has RAID/redundancy/backups configured for all of the data I store on there, which means I don't need to worry about any of that any of the clients. Sure, there are other solutions, but it was just so simple and smooth as it was.

Got it.