Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 01-18-2013, 05:17 AM   #1
Joan M
Did someone say Python?
Joan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughing
 
Joan M's Avatar
 
Posts: 51
Karma: 63486
Join Date: Nov 2012
Location: USA
Device: Kindle, Kindle Paperwhite, Nook Tablet
Question Dazed and Confused - Corrupted DB/"Lost" Books

So out of the blue (or so it seemed to a newbie) I got a message about corrupted files and did I want to attempt to restore my DB. Well, yeah. So I punched the option, "Duh, sure, let's do it."

After many hours (12? More? Dunno. I gave up and went to bed), the following message appeared:

calibre, version 0.9.14
WARNING: Success: Restoring the database succeeded with some warnings click Show details to see the details. The old database was saved as: E:\Ebooks\CalibreBooks\metadata_pre_restore.db

Failed to restore the books in the following folders:


The unrestored books appeared to represent 407 MB, 1200 files, 549 folders, maybe upwards of 400 books. And also were in the beginning alphabetically of the calibre folders - like up through "Adam Robert."

Okay. So far my questions were: (1.) Sweet Mother of Pearl! Why did this happen? (2.) Does this happen often? (3.) How should I go about restoring my 400 books.

But then I open calibre and discover that my former total of approx. 60,000 titles is now down to 36,566. This isn't fun. I've dried my tears now, but I'm not sure what I should do next.

Joan M is offline   Reply With Quote
Old 01-18-2013, 08:24 AM   #2
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,702
Karma: 5643657
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by Joan M View Post
So out of the blue (or so it seemed to a newbie) I got a message about corrupted files and did I want to attempt to restore my DB. Well, yeah. So I punched the option, "Duh, sure, let's do it."

After many hours (12? More? Dunno. I gave up and went to bed), the following message appeared:

calibre, version 0.9.14
WARNING: Success: Restoring the database succeeded with some warnings click Show details to see the details. The old database was saved as: E:\Ebooks\CalibreBooks\metadata_pre_restore.db

Failed to restore the books in the following folders:


The unrestored books appeared to represent 407 MB, 1200 files, 549 folders, maybe upwards of 400 books. And also were in the beginning alphabetically of the calibre folders - like up through "Adam Robert."

Okay. So far my questions were: (1.) Sweet Mother of Pearl! Why did this happen? (2.) Does this happen often? (3.) How should I go about restoring my 400 books.

But then I open calibre and discover that my former total of approx. 60,000 titles is now down to 36,566. This isn't fun. I've dried my tears now, but I'm not sure what I should do next.

No it does not normally happen for a standard PC configuration.
Someone was Changing folder or File names inside the Library?

Carefully, not changing anything you see: Peek inside the Library folder.
Is there a Author folder that Calibre does not show?
Is the a Title folder inside?
Is there any files inside(Normaly cover.jpg, metadata.opf, 1 or more book formats)

Is the Library on a fixed (internal) drive?
Your drive may be failing (bad Blocks)
You have a rogue Anti-virus program (you should exclude your Library, the books would normally be checked before import)

Rebuild scans the library folders for the metadata.OPF file Calibre made.
If that is missing, rebuild may not recover all the books CALIBRE metadata
theducks is offline   Reply With Quote
Old 01-18-2013, 09:49 AM   #3
Joan M
Did someone say Python?
Joan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughing
 
Joan M's Avatar
 
Posts: 51
Karma: 63486
Join Date: Nov 2012
Location: USA
Device: Kindle, Kindle Paperwhite, Nook Tablet
Quote:
Originally Posted by theducks View Post
No it does not normally happen for a standard PC configuration.
Someone was Changing folder or File names inside the Library?
Noooooooo. Never! I don't know from nothin' to be doing such.
Quote:
Carefully, not changing anything you see: Peek inside the Library folder.
Is there a Author folder that Calibre does not show?
Is the a Title folder inside?
Is there any files inside(Normaly cover.jpg, metadata.opf, 1 or more book formats)
There are many, many like that.

E:\Ebooks\CalibreBooks
Size: 59.8 GB
Size on Disc: 60.2 GB
184,572 Files, 79,250 Folders
There are 17,569 top level folders (for want of a better word) - the ones I see under CalibreBooks without opening anything up. But, again, only 36,556 total titles showing up in calibre.

Quote:
Is the Library on a fixed (internal) drive?
A 500 GB external drive gifted to me. This is actually the second one I've used. Both probably have a lot o'miles on them. The first was giving me delayed write fail error messages so I switched to the current one. The day before this corruption event, I got:

calibre, version 0.9.14
ERROR: Unhandled exception: <b>DatabaseException</b>:disk I/O error
RemoteTraceback (most recent call last):
File "site-packages\calibre\library\sqlite.py", line 271, in run
File "site-packages\calibre\library\sqlite.py", line 166, in get
OperationalError: disk I/O error


So, maybe a drive problem like you said?

Quote:
You have a rogue Anti-virus program (you should exclude your Library, the books would normally be checked before import)
I have been using Microsoft Security Essentials for anti-virus and Malwarebytes Anti-Malware uneventfully for the four months I've been running calibre.
Joan M is offline   Reply With Quote
Old 01-18-2013, 10:18 AM   #4
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,702
Karma: 5643657
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
See that line: Disk IO error
If you know how: Look in the Windows Event Viewer System section (red items) for any io/drive events

Check that all cables are snug and secure and placed so they don't get bumped.

Run the (windows) Check disk NOW. This may be your warning call . DO NOT DEFRAG A FAILING DRIVE. That is like having a Cardiac patient run up stairs. IMHO I would also (temporarily?) exclude this drive from Anti-virus scanning as that is also hard on drives (and may not be needed if no executable programs are there) .
theducks is offline   Reply With Quote
Old 01-18-2013, 10:54 AM   #5
HarryT
eBook Enthusiast
HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.HarryT ought to be getting tired of karma fortunes by now.
 
HarryT's Avatar
 
Posts: 63,656
Karma: 41565251
Join Date: Nov 2006
Location: UK
Device: PW2, iPad Retina Mini, iPhone 4, MS Surface Pro, Onyx T68, N7,
Restore your backup. This is why you have a backup.
HarryT is offline   Reply With Quote
Old 01-20-2013, 11:54 PM   #6
Joan M
Did someone say Python?
Joan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughing
 
Joan M's Avatar
 
Posts: 51
Karma: 63486
Join Date: Nov 2012
Location: USA
Device: Kindle, Kindle Paperwhite, Nook Tablet
Okay, I didn't find any I/O event items in red or otherwise.
I ran chkdsk F and then R for good measure. It fixed "errors in the uppercase file" and ended with:

488384000 KB total disk space.
369978700 KB in 207818 files.
108888 KB in 85175 indexes.
0 KB in bad sectors.
379216 KB in use by the system.
65536 KB occupied by the log file.
117917196 KB available on disk.

4096 bytes in each allocation unit.
122096000 total allocation units on disk.
29479299 allocation units available on disk.

I don't understand why I can see folders with books and calibre doesn't show that stuff. Can you explain as simply as possible how to fix this? I appreciate your help.
Joan M is offline   Reply With Quote
Old 01-21-2013, 10:06 AM   #7
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,702
Karma: 5643657
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by Joan M View Post
Okay, I didn't find any I/O event items in red or otherwise.
I ran chkdsk F and then R for good measure. It fixed "errors in the uppercase file" and ended with:

488384000 KB total disk space.
369978700 KB in 207818 files.
108888 KB in 85175 indexes.
0 KB in bad sectors.
379216 KB in use by the system.
65536 KB occupied by the log file.
117917196 KB available on disk.

4096 bytes in each allocation unit.
122096000 total allocation units on disk.
29479299 allocation units available on disk.

I don't understand why I can see folders with books and calibre doesn't show that stuff. Can you explain as simply as possible how to fix this? I appreciate your help.
The path must match exactly. Calibre is still case sensitive, even if Windows isn't. (I included a screen shot of the DB entry)

Which doesn't answer why Calibre could not recover your Library by reading the OPF file within the folder. (I think, Recover uses the current path of where it found the OPF. The OPF does not contain location.)
Attached Thumbnails
Click image for larger version

Name:	PathNames.jpg
Views:	78
Size:	158.8 KB
ID:	99829  
theducks is offline   Reply With Quote
Old 01-21-2013, 02:20 PM   #8
Joan M
Did someone say Python?
Joan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughing
 
Joan M's Avatar
 
Posts: 51
Karma: 63486
Join Date: Nov 2012
Location: USA
Device: Kindle, Kindle Paperwhite, Nook Tablet
I am computer illiterate. Are you saying I cannot use calibre to manage my ebooks? Obviously, I want to be able to work with all my files, not just some of them.
Joan M is offline   Reply With Quote
Old 01-21-2013, 02:40 PM   #9
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,702
Karma: 5643657
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by Joan M View Post
I am computer illiterate. Are you saying I cannot use calibre to manage my ebooks? Obviously, I want to be able to work with all my files, not just some of them.
Calibre works fine if you let it (and avoid a few gotchas, most which usually only bite power users)

1)never change any file/folder name inside the Library (in general, just stay out), Calibre can deliver and rename files using templates and plugboards without you knowing where they are stored.

2) Googles cloud storage is incompatible. Period. Others, such as Dropbox have been successfully used (but are not officially sanctioned).

3) Be wary of any program (anti virus is a common one) that silently fixes files believed to be in trouble. Make the setting to ALWAYS ask You (and think about it before you decide either way)


(More for a power user
Using a single Library with Multiple Os can lead to file name Case issues.
Accessing the Same (shared) Library from 2 computers at the same time can lead to corruption of the DB.
theducks is offline   Reply With Quote
Old 01-21-2013, 02:43 PM   #10
Joan M
Did someone say Python?
Joan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughing
 
Joan M's Avatar
 
Posts: 51
Karma: 63486
Join Date: Nov 2012
Location: USA
Device: Kindle, Kindle Paperwhite, Nook Tablet
Are you saying the 30.000+ titles that I can no longer see in calibre are "lost" forever?
Joan M is offline   Reply With Quote
Old 01-21-2013, 02:51 PM   #11
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,702
Karma: 5643657
Join Date: Aug 2009
Location: (The original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
I can't say one way or the other.
I can't see your file system to determine if the books are intact and it is just a DB issue.

You could Start a NEW Library in a NEW location (Library icon Menu:Switch/Create), then try an ADD Books: using the One Book per directory, Different Format option The starting folder is Inside the OLD Library
theducks is offline   Reply With Quote
Old 01-22-2013, 07:04 AM   #12
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 3,504
Karma: 2177990
Join Date: Mar 2012
Location: NSW Australia
Device: none
Joan M - what version of Windows are you using?

Is it possible that your 'gifted' disk was formatted on different version of Windows?

If XP and Yes - then that's the cause of the "errors in the uppercase file" see http://support.microsoft.com/kb/940830 - nothing to worry about, and its been rectified.

Otherwise - Hmmmm

BR
BetterRed is offline   Reply With Quote
Old 01-22-2013, 08:25 AM   #13
piperclassique
A garbling groftpot
piperclassique ought to be getting tired of karma fortunes by now.piperclassique ought to be getting tired of karma fortunes by now.piperclassique ought to be getting tired of karma fortunes by now.piperclassique ought to be getting tired of karma fortunes by now.piperclassique ought to be getting tired of karma fortunes by now.piperclassique ought to be getting tired of karma fortunes by now.piperclassique ought to be getting tired of karma fortunes by now.piperclassique ought to be getting tired of karma fortunes by now.piperclassique ought to be getting tired of karma fortunes by now.piperclassique ought to be getting tired of karma fortunes by now.piperclassique ought to be getting tired of karma fortunes by now.
 
piperclassique's Avatar
 
Posts: 730
Karma: 4793500
Join Date: Feb 2012
Location: France
Device: IPad, Kindle PW, iPhone
Joan, do you still have the original files? Or, better yet, a backup of your library?
If not, and you can see the books even if calibre can't, you can add them back in. Theducks is giving good advice. There are ways of letting other people look in your computer at a distance, if you know someone computer literate who is willing to help.
Courage!
piperclassique is offline   Reply With Quote
Old 01-24-2013, 01:47 AM   #14
derangedhermit
Addict
derangedhermit ought to be getting tired of karma fortunes by now.derangedhermit ought to be getting tired of karma fortunes by now.derangedhermit ought to be getting tired of karma fortunes by now.derangedhermit ought to be getting tired of karma fortunes by now.derangedhermit ought to be getting tired of karma fortunes by now.derangedhermit ought to be getting tired of karma fortunes by now.derangedhermit ought to be getting tired of karma fortunes by now.derangedhermit ought to be getting tired of karma fortunes by now.derangedhermit ought to be getting tired of karma fortunes by now.derangedhermit ought to be getting tired of karma fortunes by now.derangedhermit ought to be getting tired of karma fortunes by now.
 
Posts: 233
Karma: 1262174
Join Date: Oct 2010
Location: USA
Device: None
MobileRead might have 10,000 unique titles in its library.

Project Gutenberg has about 40,000 titles - Project Gutenberg!

And you have, or had, 60,000 titles?

We have to keep the pet hair blown away from our PC internals or it accumulates, causing problems. Do you keep cats?
derangedhermit is offline   Reply With Quote
Old 01-26-2013, 03:07 AM   #15
Joan M
Did someone say Python?
Joan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughingJoan M can shake the floor when laughing
 
Joan M's Avatar
 
Posts: 51
Karma: 63486
Join Date: Nov 2012
Location: USA
Device: Kindle, Kindle Paperwhite, Nook Tablet
Hey folks, sorry to ignore people who are trying to help. I just got frustrated and unplugged the SOB and decided to read books for awhile instead of $#@^ with files and software.

BetterRed - I'm on XP. sp3. Re formatting on the HDD, I was told, "Probably Win 7 (but maybe XP). Should have been formatted as NTFS (which has not changed for the last ten years or so)." But I was getting delayed write fail messages on that drive, so transferred everything over to another identical empty (used) drive. I don't even remember formatting that drive, but I must have in order to use it, right?

piperclassique - Backup? Moi? No, but lesson learned. I think I have the original files still, but who knows if the 184,572 files Windows says I have is what I started out with. I see the suggestion for " One Book per directory, Different Format option" and don't have a clue. Why would I want a different format? calibre isn't exactly intuitive for me. I suppose I shall get it sorted out one way or the other eventually.

derangedhermit - Cats? Well, sure. But I thought it must be the pythons causing the problems. The hatchlings which crawl in the USB ports to stay warm.
Joan M is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
K2 Dazed and confused. freewheeling Kindle Developer's Corner 41 11-18-2012 02:09 PM
KT "Ghost covers/files" again at 670 books, "stale" image entries in firmware VirgoGirl Kobo Reader 4 04-06-2012 02:10 PM
Yep. It's official. Sony Reader has "ruined" books for me. A final "review." WilliamG Sony Reader 48 01-14-2011 03:49 AM
Dazed and Confused (by all the options, Help!) Delia575 Which one should I buy? 21 12-01-2009 08:06 PM
5%, or 100,000 copies, of "The Lost Symbol" were sold as e-books sforce News 36 09-28-2009 06:14 PM


All times are GMT -4. The time now is 08:57 PM.


MobileRead.com is a privately owned, operated and funded community.