„Export all your calibre data“ —> Folder —> Cloud —> safe?

[vigrom]

Hello all!

I have read here in several threads that a Calibre library backup in a cloud storage is not a good idea at all and can lead either to data loss in the library or make it useless for recovery, or both (Google Drive has been mentioned here oftentimes as being particularly notorious).

Am I so far right?

If so, my question is this:

If I use Calibre’s function „Export all your calibre data“ (the one that creates several ca. 1 GB data chunks like part-xxxxx.calibre-data in a separate directory/folder or elsewhere externally), would it be safe to upload THIS folder to a cloud and then use it - if need be - as a reliable backup source of the main Calibre library?

Thank you!


PS: I understand that the contents of the folder containing part-xxxx.calibre-data chunks has to be renewed periodically und uploaded to its designated cloud to keep it there up-to-date.

[HarryT]

Originally Posted by vigrom

Hello all!

I have read here in several threads that a Calibre library backup in a cloud storage is not a good idea at all and can lead either to data loss in the library or make it useless for recovery, or both (Google Drive has been mentioned here oftentimes as being particularly notorious).

Am I so far right?

It's entirely safe to back up your Calibre library to a Cloud drive (or anywhere else) provided that Calibre isn't actually running at the time you make the backup.

What's not safe is to use a Cloud drive to host your Calibre library - ie to run Calibre with its library in the Cloud.

[vigrom]

Thank you so much, Harry!

[theducks]

Except for Google Drive. Calibre and GD both use a similar naming convention filename(#) which totally messes calibre.

You can ZIP/RAR your Library, THEN store and RETRIEVE as a whole file safely, just not unpacked

[HarryT]

Thanks for the correction, Ducks. I wasn't aware of that!

[vigrom]

Thank you for your input, Ducks!

Originally Posted by theducks

You can ZIP/RAR your Library, THEN store and RETRIEVE as a whole file safely, just not unpacked

Are the "part-xxxx.calibre-data" chunks "packed" enough to be safely up-/downloaded to/from a cloud to be used - if need be - to losslessly restore the main library?

Thank you!

[BetterRed]

Originally Posted by vigrom

PS: I understand that the contents of the folder containing part-xxxx.calibre-data chunks has to be renewed periodically und uploaded to its designated cloud to keep it there up-to-date.

Many users backup there library folder(s) and their configuration folder using a file synchronisation tool to maintain a mirror copy, locally on a usb drive and/or in the 'cloud'. Has a couple of advantages -

• after initial copy, it is faster because only changes are sent;
• restore of single items (books, formats) is possible;
• some tools can maintain an archive of changes including deletions;
• facilitates a common approach for backup of all data, rather than different solutions depending on type of data - e.g. books, images, audio, video, tax records etc etc;
• local mirror copies can be used directly via the Create/switch library feature.

Here's a list of tools ==>> File synchronisation software - Wikipedia

I use GoodSynch on a daily basis across all my data. From other users I've seen mention of: Free File Synch, rsynch, and Synch Toy.

Aside : calibre's export/import feature was implemented to facilitate migrating calibre libraries and configuration data between an old computer and a new one. In that scenario the export, and subsequent import would be done within a short period of time as a one off exercise.

See FAQ items ==>>How do I move my calibre data from one computer to another? and How do I backup calibre?

BR

[vigrom]

Thank you very much for your detailed reply, BetterRed!

Yes, I already use a powerful synch tool (Beyond Compare) for the very same goals you are describing. And yes, I also maintain a mirror copy of the library on an external HD.

Originally Posted by BetterRed

Aside : calibre's export/import feature was implemented to facilitate migrating calibre libraries and configuration data between an old computer and a new one. In that scenario the export, and subsequent import would be done within a short period of time as a one off exercise.

Still not quite clear to me… Is the folder with the data stored in it via Calibre’s export feature good and safe enough to be uploaded to and updated on a cloud to be used as a reliable backup source in case of crisis?

Many thanks!

[BetterRed]

@vigrom -- I prefer to avoid application specific features that can be done using a generalised systems management tool, such as backup/restore.

IMO it should be at least as easy to restore/access a 'backup' as it was to create it, and preferably easier. It's inevitable that the user is stressed, hence prone to making mistakes.

Originally Posted by vigrom

Yes, I already use a powerful synch tool (Beyond Compare) for the very same goals you are describing. And yes, I also maintain a mirror copy of the library on an external HD.

I know Beyond Compare as a file comparison tool, but not as as a file synch tool. Not sure BC had syncing when I decided on GoodSync - which was about a decade ago.

But if you want a backup copy of your libraries in the cloud as well as on a local usb drive (as I do), why wouldn't you use the same tool for both destinations (as I do).

Originally Posted by vigrom

Still not quite clear to me… Is the folder with the data stored in it via Calibre’s export feature good and safe enough to be uploaded to and updated on a cloud to be used as a reliable backup source in case of crisis?

Providing the machine you're restoring to has a working version of calibre that supports its import/export feature it should be OK.

But, if in this hypothetical crisis... like your house burns down... you wanted to revert to an old XP machine, then you'd have to install calibre 1.48, which doesn't have the import/export feature.

Or you might decide not to replace the desktop for a while, then you'd probably want to find a way of 'converting' calibre's exported spanned archive to something that an android or ios e-reader app could access. If the cloud backup were a synchronised mirror you could probably access it via something like iExplorer in ios, or one of the myriad of similar android apps, such as ES File Explorer.

BR

[vigrom]

@BetterRed

Oh yes, Beyond Compare does both the comparison AND the synching (+ a bunch of other useful things) and does it very well, I’d like to add.

Originally Posted by BetterRed

But if you want a backup copy of your libraries in the cloud as well as on a local usb drive (as I do), why wouldn't you use the same tool for both destinations (as I do).

Mostly because of what I read here about NOT to use clouds (Google Drive in the first place) as storage AND synching place for the main calibre library - reportedly, due to library data corruption occurring in the process of cloud-synching which then makes the uploaded version useless for restoring the library from there.

Hence, I was thinking of a „packed“ version of the library (e. g. those part-xxxx.calibre-data chunks when using calibre’s „Export…“ feature) in the hope they would be less prone to corruption when uploaded to and synched with a cloud.

(I have also learned here that the a. m. data corruption was more often than not user-induced: running calibre AND synching to a cloud at the same time. I hope I got it right.)

Providing the machine you're restoring to has a working version of calibre that supports its import/export feature it should be OK.

Yes, that is what was on my mind: to be able to restore the main library (if need be) actually onto the same machine on which it got messed up for one reason or another.

… to an old XP machine, then you'd have to install calibre 1.48

No XP machine in my case, I am on a Mac ;-)


Many thanks again!

[theducks]

Originally Posted by vigrom

Thank you for your input, Ducks!





Are the "part-xxxx.calibre-data" chunks "packed" enough to be safely up-/downloaded to/from a cloud to be used - if need be - to losslessly restore the main library?

Thank you!

Yes, those are (spanned zips) fine as long as you keep all the pieces together.

[vigrom]

Thank you!

[HarryT]

Originally Posted by vigrom

Mostly because of what I read here about NOT to use clouds (Google Drive in the first place) as storage AND synching place for the main calibre library - reportedly, due to library data corruption occurring in the process of cloud-synching which then makes the uploaded version useless for restoring the library from there.
(I have also learned here that the a. m. data corruption was more often than not user-induced: running calibre AND synching to a cloud at the same time. I hope I got it right.)

That's correct. Calibre is not "multi-user-friendly". Bad Things can happen if you do something to a Calibre library that's on a Cloud drive at the same time that the Cloud itself is doing something. Syncing your library to a Cloud drive is fine (Google Drive excepted, apparently ). Running FROM a Cloud drive is not.

[vigrom]

So noted. Thank you, Harry!