No available formats - epubs disappeared?

[mariekes]

I have been using Calibre for 4 years, some books I already have from the beginning, and never had any problems.

Today I added new books (epub), downloaded metadata and transferred them from my "work in progress" database to my actual "finished" library.
For all these books, and some others which have been in the "finished library" for years I get the message "no available formats" when trying to open the book.

When opening the path, the folder appears empty, only a metadata.opf file.
This is the case for all the books I copied from one library to the other today. But it also occurs with a lot of books that I haven't touched in years.

I have checked library maintenance /check library: no errors found
Also, never got any error messages to indicate an instable database.

Can someone please help me what's going on??
I run calibre from the local harddrive, Win7 x64, calibre 2.53 (64bit)

[theducks]

Was that folder also replicated in the Cloud?
If all you have in a title folder is the cover and OPF,:

then something told the formats to go away (delete formats
or you played 'tag' with your cloud sync
You renamed them, Calibre moves those, and the cloud tried to put them back (always pause, cloud sync)

[MartyTX]

mariekes,

Have you looked in your Recycle Bin for misplaced EPUB's?

[mariekes]

Thank you for your suggestions! I'd hate to loose my books :-)

@theducks: I'm not sure what you mean by the cloud.
All my libraries are on the local harddrive in my pc. I do not store them in dropbox, cloud or network location (only backup on external HD and network drive once in a while).
Every individial book folder only contains cover and metadata.opf. The epub has vanished?

What I did was the following:
in "work in progress library:
-for individual books sort out and delete duplicates, keep only one copy
-download metadata and bookcovers per book
-all epubs could be opened, looked just fine, no errors
-some bulk metadata editing: set language, update title case and title sort
-copy books to my "real" library, using option delete after copy
These books are all faulty.

I did the same for books in another temporary library, and moved them to the same real library, and these seem fine.

Some books in my "real" library that I haven't touched in years (in Calibre anyway) have also disappeared. Oh, that may not be completely true, for a few I did change the series name.

@MartyTX: the recycle bin did not contain misplaced epubs, only the duplicates I intentionally deleted (the entire folder is in the bin).

Any ideas?

[BetterRed]

@mariekes - suggestion - download Nir Soper's Search my File's and get it to look for epubs on your hard drives, its very fast doesn't need to be installed ==>> Alternative to Windows Search For Files + Duplicates Search

But - if you inadvertently did a Remove selected formats (Alt/Del) when you had your entire library selected, and you selected EPUB from the formats list then all the EPUBS would be trashed - i.e. they wouldn't be in the recycle bin.

BR

[mariekes]

@BetterRed: I did the search, but no luck.... it finds the folders, not the individual epubs.
I am definately sure that I didn't remove epub format from the whole set.

Any other suggestions? I am at a loss... :-(

[BetterRed]

Originally Posted by mariekes

@BetterRed: I did the search, but no luck.... it finds the folders, not the individual epubs.
I am definately sure that I didn't remove epub format from the whole set.

Any other suggestions? I am at a loss... :-(

You could try one of the file recovery utilities like Recuva and or NTFSUndelete. Beyond that you're looking at taking the disk along to a specialist data recovery crew - who don't come cheap.

I have to ask - where's your library backup?

BR

[theducks]

Originally Posted by mariekes

@BetterRed: I did the search, but no luck.... it finds the folders, not the individual epubs.
I am definately sure that I didn't remove epub format from the whole set.

Any other suggestions? I am at a loss... :-(

Did these books Originally have a cover showing in the Library (view)?
If so, that (cover.jpg) should also still be in the book folder even IF an accidental 'Remove formats' was done

Cats, Select All and Keyboards, are a dangerous combination. Never leave an ALL Selected screen for any longer than absolutely necessary

[MartyTX]

Originally Posted by mariekes

@BetterRed: I did the search, but no luck.... it finds the folders, not the individual epubs.
I am definately sure that I didn't remove epub format from the whole set.

Any other suggestions? I am at a loss... :-(

Could try expanding the search with "wild card" characters. If the .EPUB extension had a prefix or suffix added, this will find it. In gambling terms, this is a "long shot"!

[mariekes]

@MartyTX: Thanks for the picture, this helped! I did the search exactly as indicated, and it finds 44 files. Yet these do not contain any of the lost books.
To be quite honest, I am not sure what these results mean? I have 5000+ books in my library. So what's up with these 44?

@theducks: the books do still have a cover showing in the library view. I can open the jpg in the book folder. Also the metadata files are still there and can be opened with notepad.

@BetterRed: I'd probably have to start over from my backup then... Then I would just have lost a lot of time sorting out my library. The HD is somewhere in a box, recently moved to a new home :-)


Question:
If the metadata and correct covers are still there, could I just copy the epub file from my backup directly into the book folder? Or should I go through the entire process of importing the books and redoing all editing actions?

Is there anything I can do to prevent this from happening again? Because I have no idea what happened (I wish I had a cat to blame ;-)

Thanks a lot, guys!!
Happy easter!

[theducks]

Originally Posted by mariekes

@theducks: the books do still have a cover showing in the library view. I can open the jpg in the book folder. Also the metadata files are still there and can be opened with notepad.
Question:
If the metadata and correct covers are still there, could I just copy the epub file from my backup directly into the book folder? Or should I go through the entire process of importing the books and redoing all editing actions?

Thanks a lot, guys!!
Happy easter!

NEVER Add/ Remove/ Rename anything in a Library (sub) folder using tho OS. The DB is unaware you messed below the skirts.

1)since the Library Maintenance said all was good.. That indicates somethin 'officially' deleted formats and left behind what we call an 'Empty Book' (AKA just a Catalog)

2)If your metadata (and covers) have a lot of TLC applied, you will probably want to avoid starting over

3)I would go the tedious, but preserve #2, route... Drag and drop each book (from the OS explorer) into the details pane for that book. This UPDATES the existing record to include the fact there is NOW a book format associated
Caution: You can drop ANY book there. Be sure it is the CORRECT BOOK

[MartyTX]

Originally Posted by mariekes

@MartyTX: Thanks for the picture, this helped! I did the search exactly as indicated, and it finds 44 files. Yet these do not contain any of the lost books.
To be quite honest, I am not sure what these results mean? I have 5000+ books in my library. So what's up with these 44?

On my PC, the search finds several .ORIGINAL_EPUB files that wouldn't show up with a standard "epub" search. I was hoping that the wayward books would have shown up on your PC as .ORIGINAL_EPUB's. Don't know why you found only 44 files. However, I use an older version of SearchMyFiles that might search differently.

Sorry for the exercise in futility.

[BetterRed]

@mariekes - the rationale behind my suggestion to run something like Recuva was get an indication that the EPUB's have been permanently deleted. Recuva and NTFSUndelete will usually find fragments of such files and report them by name - if they didn't find anything that would suggest to me that they're hiding somewhere - like another drive.

I just did a shallow scan with Recuva over 5 drives - took about 5 minutes, a deep scan would take a lot longer



After the scan I got Recuva to search for *.txt files, the ones with a green dot are recoverable. Best practice is to recover them to another disk, if you recover them to the disk where they were then they might overwrite another recoverable file.

BR

[mariekes]

Thank you all for your help guys!

@MartyTX: I don't think the exercise was futile, it might just as well have worked :-) I did the search again, but do not find any .original_epub files.

@BR: I ran Recuva. It has found a good deal of the lost books. But unfortunately most are tagged unrecoverable because they have been overwritten :-(
Most books appear to be in the recycle bin, but some are in an unknown location D:\?\ What does that mean?


For those with "a green dot" can the epub simply be "recovered" by right-click and then magically all is well? :-)

@theducks: I have tried your suggestion. When I drag an epub from the explorer window to calibre (either book detail panel on the right-hand side in the main calibre window or the book-format box at the top-right in edit metadata, a +copy appears but nothing actually happens?

[theducks]

Originally Posted by mariekes

Thank you all for your help guys!

@MartyTX: I don't think the exercise was futile, it might just as well have worked :-) I did the search again, but do not find any .original_epub files.

@BR: I ran Recuva. It has found a good deal of the lost books. But unfortunately most are tagged unrecoverable because they have been overwritten :-(
Most books appear to be in the recycle bin, but some are in an unknown location D:\?\ What does that mean?

Attachment 147627
For those with "a green dot" can the epub simply be "recovered" by right-click and then magically all is well? :-)

@theducks: I have tried your suggestion. When I drag an epub from the explorer window to calibre (either book detail panel on the right-hand side in the main calibre window or the book-format box at the top-right in edit metadata, a +copy appears but nothing actually happens?

D: is normally a removable (CD/DVD) drive for most off the shelf PC's
(a bunch of us have 2 or more internal HD, So I also have a D: )
Did you try and restore or something from a CD/DVD? Or did you fix a Missing CD drive? This might be a letter shift that can happen under certain conditions.

Thumb rule for (default assignment) letters: Floppies (A, B), Internal HD start at C:, Next comes other Internal drives (CD, DVD, Zip ), Next comes removable drives (SD cards), External detachable (USB/FW) drives <<A device that appears as a drive, might fall here.