Disappearing temp files in Windows

[Phssthpok]

Recently, while editing several different books on Windows 10, I've hit "save" and been greeted by an error message tell me that META-INF/container.xml in the temp directory (somewhere in AppData/Local) is not found, and the file is then basically toast. Sometimes it's just the container.xml that's vanished, which I can manually recover (but it involves unzipping what's left of the epub and adding it manually, which is an annoying and long-winded process) but sometimes all the images are gone as well. Presumably there's something that's cleaning the temp directory that wasn't there a few Windows updates ago, or anti-virus updates or something. (Anyone know what it could be?)

Is there a fix for this that anyone can suggest, e.g. specifying a different temp directory? And if I do something like that, will the temp files be deleted when I close the editor or would I have to clean them manually?

[AnotherCat]

I cannot remember the exact error messages but I occasionally experience much the same. If I recall correctly a number of people, including myself, experienced this a few years back and then it went away; it has returned for me over the last month or two.

I have not troubled myself finding a fix due to the occasional nature of the issue and because I found a simple work around that works 100% for me. When I cannot do a normal save due to the issue, in Editor I do a File>Save As to hard drive, then delete the copy in Calibre, load the "Save As" copy into Calibre then open that in Editor. It works for me with no loss of any previous edits.

[BetterRed]

Try giving calibre a dedicated location for its temporary files via the CALIBRE_TEMP_DIR environment variable, see Calibre User Manual - Environment variables

Which version of Win 10. In 1709 there's a new feature, "Controlled Folder Access", that 'stops' programs using some folder operations - like opening them! Its a means of thwarting ransomware attacks. Might be worth checking out if its enabled. I think you have to opt-in to make use of it, and I have NOT heard of it deleting anything.

However there are 3rd party tools that attempt to do 'garbage collection' of the Temp directory, wouldn't surprise me if an AV vendor or something similar added such a feature - which you have to opt-out of Ψ²

BR

[kovidgoyal]

Just specify a different temp dir one that's hopefully not auto-cleaned and you will be fine. In the next release of calibre, I have added code to use the cache dir rather than the temp dir for temp files to avoid this issue with temp file cleaners.

[AnotherCat]

Originally Posted by BetterRed

...Which version of Win 10. In 1709 there's a new feature, "Controlled Folder Access", that 'stops' programs using some folder operations - like opening them! Its a means of thwarting ransomware attacks. Might be worth checking out if its enabled. I think you have to opt-in to make use of it, and I have NOT heard of it deleting anything...

One has to opt in to "Controlled Folder Access".

[Phssthpok]

Originally Posted by AnotherCat

I have not troubled myself finding a fix due to the occasional nature of the issue and because I found a simple work around that works 100% for me. When I cannot do a normal save due to the issue, in Editor I do a File>Save As to hard drive, then delete the copy in Calibre, load the "Save As" copy into Calibre then open that in Editor. It works for me with no loss of any previous edits.

I already tried that, and container.xml had already vanished. Seems like the original "save" attempt lost it somehow.

[Phssthpok]

Originally Posted by BetterRed

Try giving calibre a dedicated location for its temporary files via the CALIBRE_TEMP_DIR environment variable, see Calibre User Manual - Environment variables
BR

OK, thanks.

[Phssthpok]

Originally Posted by kovidgoyal

Just specify a different temp dir one that's hopefully not auto-cleaned and you will be fine. In the next release of calibre, I have added code to use the cache dir rather than the temp dir for temp files to avoid this issue with temp file cleaners.

I look forward to it eagerly!

[AnotherCat]

Originally Posted by Phssthpok

I already tried that, and container.xml had already vanished. Seems like the original "save" attempt lost it somehow.

OK, maybe the issue I occasionally get is different, I'll pay more attention to the error message when it happens again - I never bother to pay it much attention as they have been few and far between and the quick work around has always sorted it for me from way back.

Regarding temp folders, the issue I (and I think everyone else) has is all the dross that collects in them that is never automatically cleaned out. I know Kovid's Calibre one will remain super-clean though .

[kovidgoyal]

Yeah calibre always deletes its temp files on exit, unless it crashes. And with the change to use the cache dir for the editor/viewer I have even added code to have those programs delete any left over temp files from a crash on startup as well.

[VirgoGirl]

Hello! I have just experienced this for the first time twice in the past two days, editing an epub with Calibre's editor and having it say: "No such file or directory: 'META-INF/container.xml'
Python function terminated unexpectedly
C:\Users\JEANNE~1\AppData\Local\Temp\calibre_0ivav e\ozej2q.epub is not a valid EPUB file (could not find opf) (Error Code: 1)"
.. when I tried to reopen it. I was simply editing the text and adding some .css to the the stylesheet.

The second time I had a problem - today - I only realised there was a problem when I tried to run the spellcheck and it threw up an error. Luckily I had a backup copy on my reader which I was able to copy over and re-edit. The re-edit was fine, no issues even though I did the same things. Yesterday I was able to reconvert the file in Calibre, replacing what had disappeared (*that* may have been my fault as I was using some broad regex search and replace commands that may have caused that issue)


I'm running V3.0 on Win10, have since July and these two instances are the first problems I've had. I did notice that after I had this error message my Temp file in Appdata was pretty clean, which makes me think Windows may have dumped temp files?

[BetterRed]

Originally Posted by VirgoGirl

[ . . . ] which makes me think Windows may have dumped temp files?

If that were true, this and every other support forum would light up like Sydney Harbour on NYE.

Most likely an over aggressive 3rd party utility - such as 'AV Scanning', 'Computer Tune-Up', or 'Garbage Collection'.

BR

[kovidgoyal]

Recent releases of calibre no longer use the temp directory for storing these types of files, because this temp file deleting behavior is becoming more widespread. So if you update, you should be fine. Or you can just set the CALIBRE_TEMP_DIR environment variable as described here: https://manual.calibre-ebook.com/cus...ment-variables

[VirgoGirl]

Originally Posted by BetterRed

If that were true, this and every other support forum would light up like Sydney Harbour on NYE.

Most likely an over aggressive 3rd party utility - such as 'AV Scanning', 'Computer Tune-Up', or 'Garbage Collection'.

BR

I don't have any of those on my computer, so it can't be that.

Thanks. Kovid, I'll update. The program and the editor are great. I've never had an issue before so I assumed I somehow corrupted it in the past 2 days.

Thanks again!

[deback]

I'm using the latest version (3.18), and it's still using the TEMP directory for temp Calibre files: C:\Users\myusername\AppData\Local\Temp

VirgoGirl - You can manually delete all the files in your TEMP directory periodically, in case anything gets corrupted. It won't hurt anything at all.