ValueError: Not a ZIP file

[gianni_fed]

I made a backup of my library and, to my amazement, Caliber has not copied a large number of books giving me this error: ValueError: Not a ZIP file.
In my experience, the ZIP format is not absolutely reliable, over the years I lost many files because I had zipped them, what I know is that if you do not put the "recovery record" when you compress a file and a "corrupt header" occurs in the zipped file, the latter is unrecoverable.
From the analysis of the corrupt epubs, as can be seen in the enclosed images, the recovery record is absent, why?
What do you know, is there a way to recover them?
Thanks and bye,
Gianni
PS. sorry, I found also another error:
File "site-packages\calibre\ebooks\metadata\epub.py", line 165, in get_zip_reader
File "site-packages\calibre\ebooks\metadata\epub.py", line 148, in __init__
File "site-packages\calibre\ebooks\metadata\epub.py", line 98, in __init__
File "site-packages\calibre\ebooks\metadata\epub.py", line 153, in open
File "site-packages\calibre\utils\zipfile.py", line 995, in read
File "site-packages\calibre\utils\zipfile.py", line 1030, in open
BadZipfile: Bad magic number for file header

[sealbeater]

As far as I know, zip files don't support recovery records.

[gmw]

Check out your files in another product. You might try Sigil - the epub editor, to see if it can make fixing the files easier. Or failing that, try 7zip or similar product (if necessary rename to give a .zip file extension), to make sure the problem lies in the file and not the program reading them. (There are some explicit requirements for zipping into an epub, it's possible the software used to generate your problem files failed to comply and so causing difficulties for software that expects epub OCF compliance. A more generic .zip file utility program may be able to recover the contents where Calibre fails.)

I've never had problems with corrupt zip files (especially in comparison to other compression formats) that could not be explained by file-transfer problems (and the last of those I had were years ago - touch wood).

[gianni_fed]

Originally Posted by sealbeater

As far as I know, zip files don't support recovery records.

Yes, but why use zip instead of rar? Just because zip is open source?
I would like to spend some money to have the almost certainty of not losing my files. Today I've lost 1041 books!
More in general, why use file compression in Calibre? Today you can have tens of TeraByes for little money, maybe there's another reason that escapes me...
Thanks and bye…
Gianni

[gianni_fed]

Originally Posted by gmw

Check out your files in another product. You might try Sigil - the epub editor, to see if it can make fixing the files easier. Or failing that, try 7zip or similar product (if necessary rename to give a .zip file extension), to make sure the problem lies in the file and not the program reading them. (There are some explicit requirements for zipping into an epub, it's possible the software used to generate your problem files failed to comply and so causing difficulties for software that expects epub OCF compliance. A more generic .zip file utility program may be able to recover the contents where Calibre fails.)

I've never had problems with corrupt zip files (especially in comparison to other compression formats) that could not be explained by file-transfer problems (and the last of those I had were years ago - touch wood).

Hello,
thanks for your answer,
I used Winrar, Winzip, Z-Zip trying to open the corrupt files, but none worked. Now I've tried Sigil, but you can see the result from the attached image!!!

I would like to ask Kovid and all the developers of this wonderful program if you can modify Calibre so that it does not compress the files but save them in folders, subfolders and so on…
Many thanks.
Gianni

[DiapDealer]

Originally Posted by gianni_fed

More in general, why use file compression in Calibre?

I'm confused. If you don't want compression or zip files, why would you choose the epub format to archive your books? It's the epub format that dictates books be zip archives, not calibre. Moreover... calibre merely facilitates the storage/viewing of (and conversion to and from) existing ebook formats.

[gmw]

Gianni, I'm not sure I fully understand your situation. Calibre is not the one responsible for defining epub as a zipped collection of files. It is just working with what epub is. If any other program has accessed those epubs then that is just as likely to be responsible for any corruption introduced (assuming the file was not corrupt when downloaded) as Calibre.

Your OP said: "I made a backup of my library and, to my amazement, Caliber has not copied a large number of books giving me this error: ValueError: Not a ZIP file." You may need to explain that: how exactly you made the backup, and how any resulting problems are thought to originate with Calibre.

[sealbeater]

Originally Posted by gianni_fed

Yes, but why use zip instead of rar? Just because zip is open source?

Probably. I prefer rar for zip for that exact reason, recovery records. Sorry you lost your books, maybe you can generate a list and recover them some other way. Good luck.

[gianni_fed]

Originally Posted by gmw

Gianni, I'm not sure I fully understand your situation. Calibre is not the one responsible for defining epub as a zipped collection of files. It is just working with what epub is. If any other program has accessed those epubs then that is just as likely to be responsible for any corruption introduced (assuming the file was not corrupt when downloaded) as Calibre.

Your OP said: "I made a backup of my library and, to my amazement, Caliber has not copied a large number of books giving me this error: ValueError: Not a ZIP file." You may need to explain that: how exactly you made the backup, and how any resulting problems are thought to originate with Calibre.

Thanks to all,
I begin to believe that the problem is in my PC, it may be in the operating system, because all the problems I have encountered in the "original" version of my library.
I backed up with the "save on disk" Calibre function.
Please, can someone help me? Thank you.
Gianni

[gmw]

Originally Posted by gianni_fed

Thanks to all,
I begin to believe that the problem is in my PC, it may be in the operating system, because all the problems I have encountered in the "original" version of my library.
I backed up with the "save on disk" Calibre function.
Please, can someone help me? Thank you.
Gianni

I'm still confused. I've never used the "Save to disk" feature because my library is all local and all backups are part of my usual backup routine. I just tried now, and it seems Save to disk saves all the files separately - so you should be able to test the files in your backup and see if they are readable. If your backup files are good you should be able to use them replace the corrupted originals - presumably delete and reimport. If your backups are not good, then you are probably stuck trying to recover your files from wherever you downloaded them originally.

I notice that the library "Export/import all your calibre data" saves everything to a single folder as one or more files of what seems to be a proprietary format (at least 7zip fails to make proper sense of it) and so you would have to use Calibre to recover any data from that folder. (Which makes testing the backup a bit more difficult: presumably create a new library and then import into it using Calibre.)

[BetterRed]

Originally Posted by gmw

I notice that the library "Export/import all your calibre data" saves everything to a single folder as one or more files of what seems to be a proprietary format (at least 7zip fails to make proper sense of it) and so you would have to use Calibre to recover any data from that folder. (Which makes testing the backup a bit more difficult: presumably create a new library and then import into it using Calibre.)

@gmw - Export/Import was implemented to facilitate one-off transfer of calibre libraries and configuration data (plugins, preferences, customisations etc) between computers, including from one platform (operating/file system) to another platform - e.g. Windows/NTFS to Linux/Ext4. Anticipated usage was when someone got a new computer, but not as a backup medium.

And with Save to Disk, the Preference settings control what is saved, e.g. there are check boxes for the cover and opf files etc.

BR

[gmw]

Originally Posted by BetterRed

@gmw - Export/Import was implemented to facilitate one-off transfer of calibre libraries and configuration data (plugins, preferences, customisations etc) between computers, including from one platform (operating/file system) to another platform - e.g. Windows/NTFS to Linux/Ext4. Anticipated usage was when someone got a new computer, not backups. [...]

I guessed that was probably the case, I mentioned mainly because I wondered whether the OP may have accidentally created something using this feature and was expecting to be able to read it in other products.

[gianni_fed]

Originally Posted by gmw

I guessed that was probably the case, I mentioned mainly because I wondered whether the OP may have accidentally created something using this feature and was expecting to be able to read it in other products.

Hello,
Since I've lost 1041 books not in the copied library, but in the original library and, while I tried to copy this library on another disk Calibre not copied several books because they had a problem!
So, I would like to come back to original problem, someone can tell me what Calibre try to do in these lines?

calibre, version 3.30.0
ERRORE: Could not open e-book: Failed to read book, E:\D_Biblio\AA.VV_\I capolavori del Giallo (56402)\I capolavori del Giallo - AA.VV_.epub click "Show Details" for more information

Traceback (most recent call last):
File "site-packages\calibre\utils\ipc\simple_worker.py", line 284, in main
File "site-packages\calibre\ebooks\oeb\iterator\book.py", line 65, in extract_book
File "site-packages\calibre\customize\conversion.py", line 244, in __call__
File "site-packages\calibre\ebooks\conversion\plugins\epub_in put.py", line 266, in convert
File "site-packages\calibre\utils\localunzip.py", line 231, in extractall
File "site-packages\calibre\utils\localunzip.py", line 218, in _extractall
ValueError: Not a ZIP file

Thanks,
Gianni

[BetterRed]

The file "I capolavori del Giallo - AA.VV_.epub" is not a valid EPUB, because its not a valid ZIP file, as others have posted - an EPUB is a rebadged ZIP file.

Calibre will only read the content of files when they're added if: a) you have configured the Add Books auto-convert feature; or b) you use the get metadata from file option. Otherwise it just copies the file.

What is the source of the EPUB files?

BR

[Divingduck]

Reading the complete thread and I'am still confused what you are doing...

a) You did a backup of your calibre library.
b) You are showing a program that tries to open a ZIP file and report a major error with opening an EPUB file.
c) You are telling that you loose +1k books by backing up the calibre database.
d) You are showing a report telling that the file you like to open isn't a valid EPUB file (calibre and Sigil).
e) And you are telling that the RAR is better than ZIP. Well, this is a more philosophic statement what have nothing to do with your fileor backup problem. In this case it is simply a book format decision and not a calibre owned decision (especially for the problem you have for now). Calibre needs only to deal with existing book formats and have no influence on format definitions.

But you don't tell exactly what you are doing. Nothing from what you wrote have to do with backup / restore so fare. There is a "not in fractional parts" story to tell. How should anyone give you a good support without knowing what exactly you are doing? By the way, it is common to tell also what calibre version and OS is involved in a problem.

The big question is, how exactly did you make the backup and how did you restore.

The error you are asking for tells you exactly what had happen if calibre tries to open the file. Your EPUB isn't a EPUB. This means either you have a corrupt file, format wise a corrupt zip archive or a non EPUB file with a wrong file extension. Calibre can't do anything with this file because of these specific information:

a) "BadZipfile: Bad magic number for file header"
b) "ValueError: Not a ZIP file"
c) Sigil shows you exactly the same problem
d) Your archive program too tells you there are major problems with the file.

This all is nothing that indicate calibre as a root cause so fare.
When something like this happen you will face out immediately tons of postings here as this is a elementary problem then.

You or your system obviously did something outside calibre with your files (in case you hadn't problems before to open this files within calibre). Any idea?

Best regards, DD