recovering a very large Calibre library after system crash

[lucask]

I’m running Calibre 8.4 on Linux and have a very large library I’ve been maintaining for over a decade (around 19k files, ~180 GB). My laptop recently crashed and my only option now is to reinstall the OS.

Using a live USB, I was able to launch Calibre and use the Export Library tool to back up the library in multiple .calibre-data parts, which I’m currently uploading to Dropbox. Before I wipe the drive, I’d like some guidance:

- If in my new Calibre install the import fails for some reason, can I manually recover the old directory structure from all the .calibre-data files?
- Is there a reliable way to verify that the Dropbox uploads are complete and uncorrupted?
- ISince I don’t have access to another machine to test the restore, is there any way to confirm in advance that the exported parts will restore correctly?

More broadly, what would you do in my position?

[PeterT]

Rather than using calibre export, seeing that you have access to the files right now, why not simply copy the entire calibre library directory tree to another USB device, along with the calibre preferences (located in .config/calibre and all its sub directories); then it's simply a matter of reinstalling calibre, restoring it's preferences and library, and then starting calibre.

[JSWolf]

Originally Posted by lucask

I’m running Calibre 8.4 on Linux and have a very large library I’ve been maintaining for over a decade (around 19k files, ~180 GB). My laptop recently crashed and my only option now is to reinstall the OS.

Using a live USB, I was able to launch Calibre and use the Export Library tool to back up the library in multiple .calibre-data parts, which I’m currently uploading to Dropbox. Before I wipe the drive, I’d like some guidance:

- If in my new Calibre install the import fails for some reason, can I manually recover the old directory structure from all the .calibre-data files?
- Is there a reliable way to verify that the Dropbox uploads are complete and uncorrupted?
- ISince I don’t have access to another machine to test the restore, is there any way to confirm in advance that the exported parts will restore correctly?

More broadly, what would you do in my position?

Don't rely on Dropbox as your only backup. And don't rely on an export as your only backup. I've read of some people having problems importing an export. Your best solution would be to get a USB stick or external hard drive of large enough size and backup your entire drive. In most cases, when you install the OS, you don't have to wipe the drive as long as the drive is OK. So you'd have a full backup and maybe no need to restore the backup.

[lucask]

Originally Posted by PeterT

Rather than using calibre export, seeing that you have access to the files right now, why not simply copy the entire calibre library directory tree to another USB device, along with the calibre preferences (located in .config/calibre and all its sub directories); then it's simply a matter of reinstalling calibre, restoring it's preferences and library, and then starting calibre.

I don't have a large enough external hard drive at hand, and I can't afford to take this into the shop. I'm really worried I might only get partial backups randomly scattered if I try copying the whole folder as you suggest. I've got the checksums of all the calibre parts, wouldn't that be less risky? I can then try downloading a couple of parts and seeing whether they match.

[PeterT]

So upload the directories directly to DropBox.

I don't believe there is any way to manually recover files from the calibe export parts.

[kovidgoyal]

Upload the calibre library folder to dropbox as well. The calibre export is of the same size approximately so if you can upload one you can upload the other. Or if for some reason you dont want to do that, use your live usb to restore the exported parts as a test.

[lucask]

Originally Posted by kovidgoyal

Upload the calibre library folder to dropbox as well. The calibre export is of the same size approximately so if you can upload one you can upload the other. Or if for some reason you dont want to do that, use your live usb to restore the exported parts as a test.

Thanks for replying. The advantage I see with the export is that I can make sure all of the parts are there, and I can test random parts with a checksum to be quite certain what I have in my backup is exactly what I have locally. I can't install the dropbox client locally, and can't upload the whole folder in one go so instead I have to drag all the subfolders into the browser in batches. If I skip something or make a mistake I would have no way of telling.

Is it possible to download just a few parts and then attempt an import?

[PeterT]

I could be snarky here and suggest that you just restore your Calibre library and configuration from the backups you make on a regular basis of your important data.

You DO have backups of your important 180 Gb / 19k eBook library, right??