MobileRead Forums - View Single Post

Arro · 10-29-2016, 07:10 PM

Hello, all!

I've an interesting problem on which I was hoping someone might shed some light.

I am running Calibre 2.70.0.0 x64 under Windows 10 on an i3 with 4GB RAM and a (nearly full) 1TB hard disk.

During my early days using Calibre a good number of years ago, not certain how Calibre defined "duplicate", I set it to import duplicates. Consequently, I ended up with a large number of books which in retrospect I must agree really were true duplicates. So, using Calibre or a plugin, I recently asked the program to delete every duplicate except the oldest. This seemed to have worked, but then I noticed that, when attempting to open some books, I was returned an error saying that the book was not available in any format. A quick direct check of the library showed at least one copy of the book was indeed there. The first step using the Calibre database check or a plugin - I'm not sure which for reasons that should become clear - came up with no database errors. The second, more extensive check returned a number of issues, some of which were "fixable" (and were fixed) while others gave the only available option as deleting the offending books. I did not want to delete any books, because at that point I might have had only one copy. Although some books might have had multiple formats, I couldn't be certain. In addition, I was not certain exactly what Calibre was proposing to delete.

Now I must confess to having broken the cardinal rule of Calibre in manipulating the Calibre Library directly.

Having noticed a large number of folders containing only a cover and an opf file with no other file, I programmatically deleted every folder containing only "cover.*" and "metadata.opf". If a parent folder became empty as a result, it was deleted recursively. I had hoped, as well, that this would get rid of some of the truly bizarre author name folders from the 2000 to 2010 decade. It seems to have done so, at least in part, but not entirely.

I expected the folder deletions would cause database issues, but Calibre popped right back up, reporting my library as having been thinned by about 50,000 books. But…I still had the "no available formats" error described above.
At that point I decided to rebuild the database using Calibre. I was aware this was going to take some time. It started roughly 5 days ago and seems to have stopped at 100% on the "Quick Start Guide".

calibre.exe and calibre-parallel.exe are still active processes, but are using only a small amount of memory with neither disk nor CPU activity. No Calibre worker processes are running.

The most pressing issue presently is the always-on-top "Restoring database" dialogue box. It gives me only the option to "Cancel". It also makes it a bit more difficult to report exactly what I did and with which plugins, because I can't retrace from the main Calibre interface. I will update this post when this issue is resolved. In the meantime, my apologies for all imprecision.

In the Calibre Library itself I have two files: "metadata.db", last modified about 30 hours ago and a much larger "metadata_pre_restore.db" last modified at the time Calibre began the database restoration days ago.

I did back up the entire library before I began thinning duplicates, so I can simply revert back to it if all goes amiss.

Does anyone have any suggestions as to what has happened and how to proceed? Is there a calibre.log or equivalent anywhere any longer?

The restore process seems finished. But…….do I press the button?

Thanks to everyone!

10-29-2016, 07:10 PM	#1
Arro Member Posts: 15 Karma: 10 Join Date: Sep 2016 Device: None	Restore Database at 100% Hello, all! I've an interesting problem on which I was hoping someone might shed some light. I am running Calibre 2.70.0.0 x64 under Windows 10 on an i3 with 4GB RAM and a (nearly full) 1TB hard disk. During my early days using Calibre a good number of years ago, not certain how Calibre defined "duplicate", I set it to import duplicates. Consequently, I ended up with a large number of books which in retrospect I must agree really were true duplicates. So, using Calibre or a plugin, I recently asked the program to delete every duplicate except the oldest. This seemed to have worked, but then I noticed that, when attempting to open some books, I was returned an error saying that the book was not available in any format. A quick direct check of the library showed at least one copy of the book was indeed there. The first step using the Calibre database check or a plugin - I'm not sure which for reasons that should become clear - came up with no database errors. The second, more extensive check returned a number of issues, some of which were "fixable" (and were fixed) while others gave the only available option as deleting the offending books. I did not want to delete any books, because at that point I might have had only one copy. Although some books might have had multiple formats, I couldn't be certain. In addition, I was not certain exactly what Calibre was proposing to delete. Now I must confess to having broken the cardinal rule of Calibre in manipulating the Calibre Library directly. Having noticed a large number of folders containing only a cover and an opf file with no other file, I programmatically deleted every folder containing only "cover.*" and "metadata.opf". If a parent folder became empty as a result, it was deleted recursively. I had hoped, as well, that this would get rid of some of the truly bizarre author name folders from the 2000 to 2010 decade. It seems to have done so, at least in part, but not entirely. I expected the folder deletions would cause database issues, but Calibre popped right back up, reporting my library as having been thinned by about 50,000 books. But…I still had the "no available formats" error described above. At that point I decided to rebuild the database using Calibre. I was aware this was going to take some time. It started roughly 5 days ago and seems to have stopped at 100% on the "Quick Start Guide". calibre.exe and calibre-parallel.exe are still active processes, but are using only a small amount of memory with neither disk nor CPU activity. No Calibre worker processes are running. The most pressing issue presently is the always-on-top "Restoring database" dialogue box. It gives me only the option to "Cancel". It also makes it a bit more difficult to report exactly what I did and with which plugins, because I can't retrace from the main Calibre interface. I will update this post when this issue is resolved. In the meantime, my apologies for all imprecision. In the Calibre Library itself I have two files: "metadata.db", last modified about 30 hours ago and a much larger "metadata_pre_restore.db" last modified at the time Calibre began the database restoration days ago. I did back up the entire library before I began thinning duplicates, so I can simply revert back to it if all goes amiss. Does anyone have any suggestions as to what has happened and how to proceed? Is there a calibre.log or equivalent anywhere any longer? The restore process seems finished. But…….do I press the button? Thanks to everyone!