Calibre deleting files after edit metadata

[BetterRed]

Originally Posted by OldYamaha

These started last night and are now affecting my 2nd windows 10/calibre 6.21.

---

<snip>

---

It was working ok until I upgraded to 6.21, then this started.
I uninstalled 6.21 then install 6.20, to the exact same reults as listed above.
I have a second PC that has 2 seperate operating systems, Windows 10 and Zorin Linux. They also Have 6.21 installed but DO NOT have this issue.

Could my "Calibre Library folder" be corrupted?

I also have a new folder in "My Documents" folder called .Calibre with 2 CALIBRE_DATA files 1 is 1 MB and 1 is 96,000 KB. I've never seen this type of folder, or these Data files before.

Is that folder hidden, and what are the names of the two files?

Originally Posted by JSWolf

If you go back and look at your errors, you will see that it shows calibre 6.20. That's where the error first appeared. You aren't running 6.21.

[code]<<ERROR: Unhandled error
!Failed to read book data from calibre library. Click "Show detailss" for more information> I click show details and copied this:
<calibre, version 6.20.0
ERROR: Unhandled error: Failed to read book data from calibre library. Click "Show details" for more information

Originally Posted by JSWolf

@OldYamaha is running Calibre 6.20. It says so in the error reports.

Yet again you haven't read the original post.

BR

[OldYamaha]

Originally Posted by Sirtel

Is OneDrive syncing your Documents folder in the background?

I don't use any cloud storage. I do have 2 NAS devices, however the calibre libraries on all 3 local machines all keep local files in the local "My Documents" folder.
As I edit each book to my liking, I copy the finished book to both NAS as backups manually. About the only thing II can see that MIGHT be locking the file change would be Windows Explorer. It might be possible that a folder attribute is set wrong, but then why was there no issue with ANY previous iteration of Calibre 6? I've used 6.2.1 on up until the upgrade to 6.21 on Tuesday night when this started.

[OldYamaha]

Originally Posted by JSWolf

@OldYamaha is running Calibre 6.20. It says so in the error reports.

If you read the error report, it says that the errors started AFTER I upgraded to 6.21. I uninstalled 21 and reinstalled 20 to see if that would correct the problem, which it didn't. I can replicate the errors on 21 on my other windows 10 pc and post those too, if you want. anything to help.

[kovidgoyal]

@OldYamaha: close windows explorer (or close whatever calibre library folder you have open in it)

[OldYamaha]

Sorry, I didn't see that the 6.21/20 issue was quoted. not trying to pile on, just trying to explain. Sorry.

[ownedbycats]

Originally Posted by kovidgoyal

@OldYamaha: close windows explorer (or close whatever calibre library folder you have open in it)

Open Task Manager, kill explorer.exe, then use file>run>'explorer.exe' to restart it. This should kill any open locks from it.

[OldYamaha]

Yes, I saw that that had been mentioned to try, so I tried it, on 4 different books, and that does work. Its a little awkward form the past, but that does seem to work.
So then, is this a windows issue or just an awkward communications issue in programming languages?
Harry

[ownedbycats]

Windows is kind of screwy when it comes to filelocks and the such.

[OldYamaha]

I just closed the open explorer views simply and that seems to have worked.
Harry

[kovidgoyal]

calibre now locks folders when copying book folders not just files. This was done because some people were getting errors caused by other programs locking folders and thereby causing the copy operation to fail halfway through.

This means you need to close windows explorer/not keep it open on a calibre library folder if you intend to change the title/author of a book. This entire issue is largely because windows file system semantics are absolutely bonkers. It is impossible for multiple unrelated programs in windows to share access to the same folder/files robustly.

[OldYamaha]

Been using Calibre since Vista, and calibre 2.28, and I now have 8 posts here so KUDOs to Kovid, he has done a wonderful job.
Harry

[JSWolf]

I've been using calibre since it was LibPRS500 and up until 6.20, I've never has a file lock issue with calibre.

Can the new locking be made an option so it can be turned off?

[JimmXinu]

I'm a bit late to this party, but I think I have another data point that may be relevant.

It looks very much like Window File Explorer search that happens to include Calibre library can cause lock issues long after the search was done.

I was recently using Windows File Explorer search--something I do very rarely--on my c:/users/USER dir, which also includes my Calibre libraries. I did my search and moved on, leaving file explorer open (I use that a lot), but not on the search anymore.

Afterwards--several hours afterwards--I was blocked from changing a book title in Calibre until rebooting.

Later, I duplicated and saw that it was explorer.exe locking the book dir in the library.

And I've again duplicated it just now:

1. Use Windows File Explorer search that happens to include Calibre library but wasn't for anything in the library.
2. Go on to something else, leaving search results, but keeping File Explorer open.
3. Calibre fails trying to rename a book.
4. Wait close to 2 hours while using computer for other tasks.
5. Calibre still fails trying to rename a book.
6. Close File Explorer
7. Calibre can successfully rename book.

This, IMO, is a lot more problematic than 'user had file explorer open in library' because it's not obviously related. Users with their libraries under Documents could bump into this easily and frequently without having any idea why it's happening.

If it weren't for the fact I use file explorer search so fairly, I doubt I'd have spotted it either.

Error message:

Spoiler:

calibre, version 6.22.0
ERROR: Cannot open file or folder as it is in use: Could not change on-disk location of this book's files. The folder "C:\Users\USER\Documents\Library\Andrew Joshua Talon\Paint and Powder (10896)" is opened in another program, so calibre cannot access it.This is usually caused by leaving Windows explorer or a similar file manager open to a folder in the calibre library. Close Windows explorer and retry.

calibre 6.22  embedded-python: True
Windows-10-10.0.19045-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19045')
Python 3.10.1
Windows: ('10', '10.0.19045', 'SP0', 'Multiprocessor Free')
Interface language: None
Successfully initialized third party plugins: DeDRM (10, 0, 3) && ColumnSum (2, 2, 0) && Count Pages (1, 13, 0) && EpubMerge (2, 15, 1) && EpubSplit (3, 6, 0) && FanFicFare (4, 24, 9) && Favourites Menu (1, 3, 0) && Find Duplicates (1, 10, 8) && Generate Cover (2, 3, 2) && Jimms Editor Addons (0, 0, 1) && Manage Series (1, 5, 1) && Modify ePub (1, 8, 1) && Quality Check (1, 13, 4) && Reading List (1, 15, 1) && ScrambleEbook (0, 6, 0) && SmartEject (2, 5, 0) && SpacerName (0, 0, 2) && SplitMergeNew (0, 3, 0) && View Manager (1, 10, 2)
Traceback (most recent call last):
  File "calibre\utils\copy_files.py", line 112, in _open_file
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\USER\\Documents\\Library\\Andrew Joshua Talon\\Paint and Powder (10896)'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "calibre\gui2\metadata\single.py", line 625, in accept
  File "calibre\gui2\metadata\single.py", line 608, in apply_changes
  File "calibre\gui2\metadata\basic_widgets.py", line 231, in commit
  File "calibre\db\legacy.py", line 934, in func
  File "calibre\db\cache.py", line 85, in call_func_with_lock
  File "calibre\db\cache.py", line 1481, in set_field
  File "calibre\db\backend.py", line 1726, in windows_check_if_files_in_use
  File "calibre\utils\copy_files.py", line 250, in windows_check_if_files_in_use
  File "calibre\utils\copy_files.py", line 136, in __enter__
  File "calibre\utils\copy_files.py", line 132, in open_all_handles
  File "calibre\utils\copy_files.py", line 123, in _open_file
  File "calibre\utils\copy_files.py", line 112, in _open_file
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\USER\\Documents\\Library\\Andrew Joshua Talon\\Paint and Powder (10896)'

[JSWolf]

Originally Posted by JimmXinu

I'm a bit late to this party, but I think I have another data point that may be relevant....

Do you think this could be related to the newly added locking in Windows added to 6.20/6.21?

[ownedbycats]

Originally Posted by JSWolf

Do you think this could be related to the newly added locking in Windows added to 6.20/6.21?

That's what I suggested in the FanficFare thread. Oddly, the discovery came after a similar issue in a very old 3.48 version (Windows 7 user).