Corrupted library recovery advice, probably a stupid question

[fsirett]

First of all, I have been a very happy Calibre user for many years. To say I love it is an understatement. If my wife would let me, I would marry this software.

Nest I am using Ubuntu 21.04 and I have upgraded to the latest version of Calibre.

Now the problem. On Saturday, I lost most access to Calibre and all of my library with a database problem. I read through these forums and took the advice and opened a new library and everything went back to normal. Great!

However my old library is very large and is not something I would choose to lose, so I went back and looked at it with other software. Every book I sampled, opened without the slightest problem. so I test transferred a few to the new library and they do not appear in the lists. To me this seems to indicate I have some, hopefully minor, corruption problems. That is the problem I have. I also tried to move them back in as a book file from the Calibre GUI and it was having none of that either.

I am thinking all I need do is go into safe mode and use "fsck" to check all of the files and then go back to that library, then I hear people saying that this method could ruin the entire file. I really do not see how, myself, but it is what I do not know that gets me every time.

So my question is this. Given that I am using the latest Ubuntu and the latest Calibre, what is the best way of repairing the corrupted file, assuming that it can be repaired.

I do know this is a very easy question and one that will be glaringly obvious, but I can only think if I am not certain and it is not answered here, then maybe it should be to save time later.

More like it has been answered many times and I am just blind or stupid.

Thanks

Fsirett

[fsirett]

Just when I thought I had it nearly beaten, I have a new problem. Let me post the message first:

calibre, version 5.22.1
ERROR: Unhandled exception: <b>CorruptError</b>:CorruptError: database disk image is malformed

calibre 5.22.1 embedded-python: True is64bit: True
Linux-5.11.0-22-generic-x86_64-with-glibc2.2.5 Linux ('64bit', 'ELF')
('Linux', '5.11.0-22-generic', '#23-Ubuntu SMP Thu Jun 17 00:34:23 UTC 2021')
Python 3.8.5
Interface language: None
Successfully initialized third party plugins: DeDRM (7, 2, 1) && Find Duplicates (1, 8, 5) && KePub Metadata Reader (3, 4, 3) && Obok DeDRM (7, 2, 1) && Reading List (1, 7, 6) && Resize Cover (1, 1, 0)
Traceback (most recent call last):
File "calibre/gui2/actions/view.py", line 295, in view_triggered
File "calibre/gui2/actions/view.py", line 372, in _view_books
File "calibre/gui2/actions/view.py", line 324, in _view_calibre_books
File "calibre/gui2/actions/view.py", line 150, in view_format_by_id
File "calibre/gui2/actions/view.py", line 134, in calibre_book_data
File "calibre/db/cache.py", line 76, in call_func_with_lock
File "calibre/db/cache.py", line 2426, in annotations_map_for_book
File "calibre/db/backend.py", line 1797, in annotations_for_book
File "calibre/db/backend.py", line 293, in annotations_for_book
File "calibre/db/backend.py", line 372, in execute
apsw.CorruptError: CorruptError: database disk image is malformed

When I tried to read my news that I had downloaded this morning, Calibre is not letting me look at them. I suspect this might have something to do with the earlier problem, or not. I am afraid my programming knowledge is very rudimentary, so I need your help...again.

F

[kovidgoyal]

You can recover a corrupted database with

calibredb --library-path=/path/to/library/folder restore_database -r

[fsirett]

I put that code into a terminal and it returned this:

Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Barnes & Noble.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Biblionet_gr.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Fantastic Fiction Adults.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Goodreads.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/KePub Metadata Writer.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Kobo Books.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Skoob.zip'
Starting restoring preferences and column metadata ... 0%
Cannot restore preferences. Backup file not found. ... 100%
Restoring database failed with error:
Traceback (most recent call last):
File "calibre/db/restore.py", line 129, in run
File "calibre/db/restore.py", line 286, in replace_db
File "shutil.py", line 261, in copyfile
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/library/folder/metadata.db'

I thought maybe I needed to prefix it with "sudo" and I got this:

Usage: calibre [options] [path_to_ebook or calibre url ...]

Launch the main calibre Graphical User Interface and optionally add the e-book at
path_to_ebook to the database. You can also specify calibre URLs to perform various
different actions, than just adding books. For example:

calibre://view-book/test_library/1842/epub

Will open the book with id 1842 in the EPUB format from the library
"test_library" in the calibre E-book viewer. Library names are the folder names of the
libraries with spaces replaced by underscores. A full description of the
various URL based actions is in the User Manual.


Whenever you pass arguments to calibre that have spaces in them, enclose the arguments in quotation marks. For example: "/some path/with spaces"

calibre: error: no such option: --library-path

I am not certain what the problem is and I do not want to push it much farther without knowing I am going down the right path.

Just thinking, I have not moved the old library back into the game. Let me do that and see if that happens to work. Another stupid oversight on my part.

Okay, first try, without the "sudo command came up with this:

Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Barnes & Noble.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Biblionet_gr.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Fantastic Fiction Adults.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Goodreads.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/KePub Metadata Writer.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Kobo Books.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Skoob.zip'
Starting restoring preferences and column metadata ... 0%
Cannot restore preferences. Backup file not found. ... 100%
Restoring database failed with error:
Traceback (most recent call last):
File "calibre/db/restore.py", line 129, in run
File "calibre/db/restore.py", line 286, in replace_db
File "shutil.py", line 261, in copyfile
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/library/folder/metadata.db'

This looks much the same as the last one, but I am having probems with my pointer, so it is difficult to compare. So it would appear the new and old libraries still have the same problem

With the "sudo command I got this:

Starting restoring preferences and column metadata ... 0%
Cannot restore preferences. Backup file not found. ... 100%
Restoring database failed with error:
Traceback (most recent call last):
File "calibre/db/restore.py", line 129, in run
File "calibre/db/restore.py", line 286, in replace_db
File "shutil.py", line 261, in copyfile
FileNotFoundError: [Errno 2] No such file or directory: '/path/to/library/folder/metadata.db'

That one is different, but much the same. It is the last line I find worrying in both cases. I know there is something that is obvious here, just not to me.

Cheers again,

F

[kovidgoyal]

You need to replace /path/to/library/folder with the actual path to your calibre library folder.

[fsirett]

Did I mention I was stupid?

[fsirett]

Okay, I was able to eventually get the path right. Using an external disk for this sort of thing is new to me and it was a touch confusing. but I did get an odd remark in the terminal:

frank@frank:~$ calibredb -- /media/home/frank/My Passport/Calibre restore_database -r
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Barnes & Noble.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Biblionet_gr.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Fantastic Fiction Adults.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Goodreads.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/KePub Metadata Writer.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Kobo Books.zip'
Failed to initialize plugin: '/home/frank/.config/calibre/plugins/Skoob.zip'
You must provide the --really-do-it option to do a recovery

the "--really-do-it" option is confusing me.

I might be dumb, but I keep careful notes so I do ot have to bother people again, if that is of any comfort.

It looks like it has sort of worked and I think it is the code you prescribed, but I might have made a whole set of new mistakes. Finding that I had used capitals in the titles had me stumped for a while.

I am pleading a hot summer day as a mitigating circumstance.

F

[kovidgoyal]

calibredb restore_database --libray-path "/media/home/frank/My Passport/Calibre" -r

[fsirett]

Sorry to be so late replying. Work got in the way.

First of all, I had problem until I noticed the spelling mistake, then it would not accept my location, so I looked again at my path and thought: "it looks like external drives are accessed through the /home folder, but possibly not. I removed the /home part of the pat, and it worked.

In case another Ubuntu user comes this way, that is at least one of the problems.

I can now access the library with little or no problem. I suspect there will be damage, but as long as there is some progress, I have to live with that.

Now, it will not download my news feeds at all. I can call up the page and hit the button but it just sits there. I find that very odd.

When I tried to change a book - PDF to EPUB - it did not work but it could be that particular book. I have to find another non-EPUB book to try again, but I have some suspicions that I would like to ask about.

I am thinking of removing and reinstalling Calibre and see if that works.

The reason I ask is that I have, in the past, tried that and found it did not work and so thought it was another corrupted file. This went on some five or mre times before I asked and found I was chasing my tail, and I don't even have a tail!

Is that a wise path to take?

Cheers,

F

[fsirett]

I just found the news works with my new library. It took a few days to kick in, probably something I did, but there you have it, the News works as before.

Calibre is my favourite software and my go to tool every day. I was in a near panic thinking I would have to change to something else. I now suspect that most of my problems with the library may have come from the News downloads.

In any case, I would like to change the status to this thread as "Solved," but I do not see any way of doing that, So I am going to put it in the title and hope that does the trick.

Many thanks for all your help,

F