Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 01-05-2009, 08:42 PM   #1
Philantrop
Addict
Philantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beauty
 
Philantrop's Avatar
 
Posts: 258
Karma: 32099
Join Date: Dec 2008
Device: iOS devices
metadata.db corrupted

I have shredded my metadata.db and it's gone.

I have, however, an sqlite3 dump of the whole database from before the problem occurred.
Thus, I can recreate metadata.db (using sqlite3 metadata.db < dumpfile) which I can verify by connecting to the db and selecting data from it which works fine.
Neither calibredb nor the GUI see any of my books, though.

Instead both claim to update the database and delete all the content while doing so:

Code:
$ calibredb list
Upgrading database to version 2...                                                   
Upgrading database to version 3...                                                   
Upgrading database to version 4...                                                   
idtitleauthors
I can't seem to find a way to stop them doing that. What am I missing and what can I do to recover my data - after all, it's all at my fingertips and yet still so far...
Philantrop is offline   Reply With Quote
Old 01-05-2009, 08:54 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,430
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
What happens if you run it with

Code:
calibre --with-library /path/to/folder/containing/recovered/metadata.db
kovidgoyal is offline   Reply With Quote
Old 01-05-2009, 09:08 PM   #3
Philantrop
Addict
Philantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beauty
 
Philantrop's Avatar
 
Posts: 258
Karma: 32099
Join Date: Dec 2008
Device: iOS devices
If I recover the db to database_path (which is on NFS if it matters) and run calibre with --with-library, it starts empty and empties the recovered db again.

If I recover the db to an alternative path, calibre starts empty as well but, interestingly, the database doesn't get wiped.
Philantrop is offline   Reply With Quote
Old 01-05-2009, 09:15 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,430
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Edit global.py

Change library_path to point to some folder on your local drive.
delete database_path
set migrated to True

Now copy your recovered metadata.db to the folder you specified earlier as well as all the actual ebook folders/files

Now run calibre

Note that library_path sets the location of the calibre library, not database_path, which is on there for legacy reasons
kovidgoyal is offline   Reply With Quote
Old 01-05-2009, 09:29 PM   #5
Philantrop
Addict
Philantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beauty
 
Philantrop's Avatar
 
Posts: 258
Karma: 32099
Join Date: Dec 2008
Device: iOS devices
library_path now points to a local dir with the recovered metadata.db inside.
migrated = True (which was already the case, though)
I've removed database_path from global.py.
I've copied the recovered (and verified as containing my data) to library_path and the NFS dir in which my actual ebooks (as stored there by calibre) reside.

Then I started calibre:

Code:
$ calibre
link  hasn't been detected!
link  hasn't been detected!
Upgrading database to version 2...
Upgrading database to version 3...
Upgrading database to version 4...
(The "link" message has always been there.)

No books are shown in calibre.
library_path/metadata.db is wiped empty after quitting calibre. The other metadata.db in the ebook storage area (how do you call that?) is still fine.

The very same setup (with everything on NFS) has worked fine till the original metadata.db got killed (manually by myself by accident).

Last edited by Philantrop; 01-05-2009 at 09:31 PM. Reason: Further information.
Philantrop is offline   Reply With Quote
Old 01-05-2009, 09:43 PM   #6
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,430
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
What happens with

Code:
calibre-debug -c "from calibre.library.database2 import LibraryDatabase2; db = LibraryDatabase2('/path/to/folder/containing/recovered/metadata.db');"
kovidgoyal is offline   Reply With Quote
Old 01-05-2009, 09:48 PM   #7
Philantrop
Addict
Philantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beauty
 
Philantrop's Avatar
 
Posts: 258
Karma: 32099
Join Date: Dec 2008
Device: iOS devices
Quote:
Originally Posted by kovidgoyal View Post
What happens with

Code:
calibre-debug -c "from calibre.library.database2 import LibraryDatabase2; db = LibraryDatabase2('/path/to/folder/containing/recovered/metadata.db');"
Upgrading database to version {2,3,4}...

And the recovered metadata.db is emptied again.
Philantrop is offline   Reply With Quote
Old 01-05-2009, 10:07 PM   #8
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,430
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
that means your recovered metadata.db is also corrupted. What size is the db file? One thing to try is the following

Run

sqlite3 metadata.db

At the sqlite prompt

pragma user_version=4;

quit and then try the previous command again
kovidgoyal is offline   Reply With Quote
Old 01-05-2009, 10:21 PM   #9
Philantrop
Addict
Philantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beautyPhilantrop does all things with Zen-like beauty
 
Philantrop's Avatar
 
Posts: 258
Karma: 32099
Join Date: Dec 2008
Device: iOS devices
First of all, Kovid, thank you very, very much. Your quick and effective help is greatly appreciated.

In the end, "pragma user_version=4;" did the job. Calibre is now working for me again, I have two independent file backups of metadata.db which is now stored in git as well - just to be sure.

Looks like sqlite3's ".dump" does not dump everything Calibre needs? Is there another way to create a "calibre-compatible" dump?
Philantrop is offline   Reply With Quote
Old 01-05-2009, 10:34 PM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 26,430
Karma: 5383257
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Glad it worked

Basically, the only thing it doesn't dump is user_version (possibly because that's not really a part of the database). At some point I have to create a backup/restore infrastructure for calibre, if only the day had 240 hours
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Corrupted Metadata Marcy Calibre 31 07-27-2010 11:50 AM
Corrupted File? StealthyScribe Kobo Reader 8 07-21-2010 11:09 PM
corrupted dbase hnoto Related Tools 43 05-17-2010 12:27 PM
Corrupted DB on Reader djwoodard Kobo Reader 7 05-10-2010 02:13 AM
Library corrupted rozie123 Sony Reader 1 05-26-2009 08:02 AM


All times are GMT -4. The time now is 07:41 PM.


MobileRead.com is a privately owned, operated and funded community.