Storing the Calibre library in "Documents Folder" on iCloud

[theducks]

Originally Posted by stumped

No one can assess the risk for you. I've lost count of the number of " help me troubleshoot this strange bug " threads where, about 20 posts in, with all contributors scratching their heads, the OP admits they were using a NAS or a cloud library.

There have also been cases reported here where " it worked perfectly for years" until one day it did not.
you are not in control of what background activites the cloud software gets up to, because it can be making its own silent backups and thus temp locking, unlocking critical files.

I would be guided by the program author's advice, but its up to you....

I often edit a book, using sigil, operating directly on the file stored in the calibre database, but that is all local, and I am the only user, and I have a backup. Been doing it for years, never any issues... [ there you go, that should bring down a curse any day now ]

Amen! I used to have as much hair as Kovid does... HAD!

Editing any Library book externally is not a problem, AS LONG AS THE FILE NAME AND PATH remain untouched. Calibre will just not update the size column value (The quality check PI: Fix: file size, will)

[ispaure]

Just thought I'd chime in, having tried this myself today (Calibre library in iCloud Drive).

TL;DR Don't do it! It doesn't behave properly

Now, because noone wants to hear "No" without reasons, let me show you what happens:
-You sync library to iCloud: All is fine and showing on iCloud.com
-You can add books, rename them, add them to your kindle device
Until... (and I've specifically stressed test this possibility)
-You run out of storage space on your Mac. If you have optimize storage turned on, this means some files of the library may only be available online on iCloud.com. You have a link in the Finder to re-download the file when you need it, but Calibre is "dumb" and just sees the file as missing.
-What may occur afterwards: Can't open books not synced from the Calibre library (must go in Finder to open them). Also, books may get deleted (I'm not sure why, but they do)

Thankfully, I did test with a copy of my own library (and lost nothing). But! Let it be a lesson to you guys: as tempting as may be, Calibre just doesn't play well with Cloud storage. It doesn't try to "get" files that are not there and depends on some stuff at the filessystem level. A quick google search of "calibre google drive" will tell you what I'm talking about.

Now, I am using calibre on my LAN, through a network share from a PC. Probably still prone to breaking, but I have a backup and will closely inspect for any changes...

[JSWolf]

Originally Posted by stan999

Well, that's why I'm asking. I'm aware of the issues. I'm making some technical judgments here. If a change inside the local library would not change metadata (as in open pdf with pdf expert, make highlights...close db) then I wouldn't see what the problem of doing a two sync be. And I now understand the main function of .opf files. ll keep them there even though I usually back up the library every day.

You do it and you could corrupt your library such that you'd have to build it again from scratch including all customizations and metadata fixes.

[JSWolf]

Originally Posted by ispaure

Now, I am using calibre on my LAN, through a network share from a PC. Probably still prone to breaking, but I have a backup and will closely inspect for any changes...

I do the same thing. I've been doing it for over 10 years without a single problem. If you are using a laptop, you do have to close Calibre before shutting the lid. And do not run Calibre more then once at a time. Other then that, you'll be good to go.

[stan999]

Thank you guys. Good information.

[ispaure]

Originally Posted by JSWolf

I do the same thing. I've been doing it for over 10 years without a single problem. If you are using a laptop, you do have to close Calibre before shutting the lid. And do not run Calibre more then once at a time. Other then that, you'll be good to go.

Great to know! Thanks for validating. As a side note, iTunes/Music deals the same way once song files are only in iCloud (showing them in the library, but won't play them unless you manually go and sync them yourself). I will have to investigate on a network-based solution for that, too. I'm curious if you'd have any tips? (I do realize this is a bit outside of the subject of this forum, though)

I was hoping for iCloud to be more similar to Microsoft's OneDrive whereas when an application requests a file, it is automatically downloaded on the backend without developers having to add any support whatsoever.

[BetterRed]

@ispaure - I just thanked you for your informative posts by dumping some points in your Karma score.

Welcome to Mobileread, and thanks again.

BR

[DNSB]

Originally Posted by stumped

I often edit a book, using sigil, operating directly on the file stored in the calibre database, but that is all local, and I am the only user, and I have a backup. Been doing it for years, never any issues... [ there you go, that should bring down a curse any day now ]

You could always open the book in Sigil and calibre's editor and then read it (with the bookmark save enabled) while editing in both editors. The results when I tried this were, ummm... interesting.

And yes, I was deliberately trying to see if I could cause issues.

[Adoby]

What makes networked storage extra nasty is that it can, in some cases, cause silent corruption over time. As you look on your library in calibre (the database) everything looks fine. But the actual book formats may go missing or stop matching the database. And you don't discover this until you try to open a book and it is missing, at some other location or corrupt. And by that time many other books may also be missing, no longer match the database or be corrupt.

[badbob001]

I also had calibre portable running on my nas due to low disk space on local drives and the nas storage is redundant and backed up. This had worked perfectly for many years but with a recent version of Calibre, the ebook viewer stopped working.

I did get more local storage so I'm going back to storing calibre locally and upon exit, have it copy any file changes up to the nas.

[stumped]

Originally Posted by DNSB

You could always open the book in Sigil and calibre's editor and then read it (with the bookmark save enabled) while editing in both editors. The results when I tried this were, ummm... interesting.

And yes, I was deliberately trying to see if I could cause issues.

i would not do that. one user, one pair of hands so I think I can be reasonably sensible and not crazily multi-task...

I did notice 1 small glitch today, running free file sync as usual ( new version, just updated)... just after adding and editing a book in calibre.
FFS started complaining about a .tmp file that it could not move. Maybe I had not just given calibre time to clean up as the file name related to the just added book.
I also got a new FFS warning ( due to new FFS version)- it didn't like that I have an FFS task that backs up calibre to 3 destinations [ local drive, dropbox, pcloud - all one way mirrors - but for one of them I had a filter set to skip all .original_epub format. It was not obvious exactly what it did not like about that so I removed the filter rather than argue with it.