calibre library/ dropbox / system image questions

[eschwartz]

I read the link, and it sounds like that answer is a total guess. The answerer has given no reason for his statement.

It could be based on checksum matching, which is what I'd do, or it *could* be based on Dropbox memorizing which computer last synced each file. Seems an awkward way of doing it.

I would prefer using my own data to answer this, what I recommend doing is using FreeFileSync to do a one-way mirror of the library, restore the partition with Dropbox and the mirror of the library, then see what happens.

For bonus points I would post an actual answer with actual facts on SuperUser.

[cybmole]

any such test would take several hours.
i do have
1. a spare PC that has only a win 7 install + some utilities, thus only about 20Gb used.
2. a spare dropbox account
I'd need to activate & populate that drop box account on that PC, image the drive via usb to external portable , then make some changes to that dropbox, then restore the drive & inspect the results.

a shorter, less conclusive test would be:
1. set up and populate that spare dropbox account on that PC as before.
2 Now, just copy the dropbox folder AND all the dropbox app data,
3 make some dropbox changes,
4 take that pc offline, restore all the copied files, put it back online, observe the results.

That would only take an hour, including retrieving that pc from the attic. I guess I'm suprised that neither test has not already been done & documented, not anywhere that I can find anyway.

I think it likely that dropbox "secret" rules work like this.
in the dropbox cloud are files plus records of which PC uploaded each file
when dropbox sees that a local file ON THE PC THAT UPLOADED IT is different to the cloud copy, then it always uploads, ( i.e. there is no datestamp check ) but when it sees that a local file on some different PC is different to the cloud copy ( AND maybe also fails a date check) then it downloads. The effects on a calibre library would be subtle, if a book is simply tweaked or edited, no file names change, i'ts just that the content of the epub files changes & presumably the timestamp. So a test for how that is treated would involve changing the content but not the name of one or more dropbox files, in the above tests

FYI I went away & read up on how google drive works - the google drive files are NOT stored locally & take up no local space ( which was news to me) your local "google drive" is just a set of pointers to cloud files.. So google drive could still get confused or corrupted by system image restore, maybe ( I need to logic it out ) , but it would not be directly comparable to the dropbox case. I have not tried operating google drive across 2 PCs.

[eschwartz]

Was thinking of doing the copy version myself.

[Purple Lady]

Originally Posted by cybmole

any such test would take several hours.
i do have
1. a spare PC that has only a win 7 install + some utilities, thus only about 20Gb used.
2. a spare dropbox account
I'd need to activate & populate that drop box account on that PC, image the drive via usb to external portable , then make some changes to that dropbox, then restore the drive & inspect the results.

a shorter, less conclusive test would be:
1. set up and populate that spare dropbox account on that PC as before.
2 Now, just copy the dropbox folder AND all the dropbox app data,
3 make some dropbox changes,
4 take that pc offline, restore all the copied files, put it back online, observe the results.

That would only take an hour, including retrieving that pc from the attic. I guess I'm suprised that neither test has not already been done & documented, not anywhere that I can find anyway.

I think it likely that dropbox "secret" rules work like this.
in the dropbox cloud are files plus records of which PC uploaded each file
when dropbox sees that a local file ON THE PC THAT UPLOADED IT is different to the cloud copy, then it always uploads, ( i.e. there is no datestamp check ) but when it sees that a local file on some different PC is different to the cloud copy ( AND maybe also fails a date check) then it downloads. The effects on a calibre library would be subtle, if a book is simply tweaked or edited, no file names change, i'ts just that the content of the epub files changes & presumably the timestamp. So a test for how that is treated would involve changing the content but not the name of one or more dropbox files, in the above tests

FYI I went away & read up on how google drive works - the google drive files are NOT stored locally & take up no local space ( which was news to me) your local "google drive" is just a set of pointers to cloud files.. So google drive could still get confused or corrupted by system image restore, maybe ( I need to logic it out ) , but it would not be directly comparable to the dropbox case. I have not tried operating google drive across 2 PCs.

Don't use Google drive for Calibre library - it's not compatible. https://www.mobileread.com/forums/sho...d.php?t=205581

[cybmole]

Originally Posted by Purple Lady

Don't use Google drive for Calibre library - it's not compatible. https://www.mobileread.com/forums/sho...d.php?t=205581

I am well aware of that already, thanks.
all I am copying is one way copying from the live calibre library ito google drive, as an additional backup.
it is perfectly safe to copy - not move - your calibre library to cloud storage.
the worry - & the purpose of this thread- is the possible pitfalls when you try to retrieve it from cloud because your primary local source is lost.

[Purple Lady]

Originally Posted by cybmole

I am well aware of that already, thanks.
all I am copying is one way copying from the live calibre library ito google drive, as an additional backup.
it is perfectly safe to copy - not move - your calibre library to cloud storage.
the worry - & the purpose of this thread- is the possible pitfalls when you try to retrieve it from cloud because your primary local source is lost.

But if your primary source is lost and you try to copy from Google drive I thought that is when the problem occurs. I also read that Google will rename the number part of the folder name to (1), but I never saw if that only happens if you update it. What I saw seemed to say it was just renamed at some point. Are all your folder names still correct?

[chaley]

Originally Posted by cybmole

it is perfectly safe to copy - not move - your calibre library to cloud storage.

Unless Google Drive has changed something, it is not "perfectly safe" to copy your library there. Google Drive thinks it can change any number that appears in parentheses at the end of the file name. If it chooses to do that to you, your backup is rendered totally useless.

[theducks]

Zip/rar your Library, THEN:
put the RAR/Zip on GD

All paren are INSIDE the zip and safe

[eschwartz]

The OP is well aware of the issue with Google Drive, as has been expressed. There is nothing wrong with using Google Drive as an additional backup for redundancy purposes. The main library will not be corrupted, and there are other backups.

And if worst comes to worst and the Google Drive backup must be used, he will simply import from the save. All metadata will be safe (that is the important thing, besides not losing the ebooks themselves to oblivion), and it is no different from running Restore Database.

Constantly rezipping the library in order to put it on Google Drive would take far more effort than it is worth for a redundant backup.

And for the sake of redundancy -- this is a redundant backup anyway.

[BetterRed]

@eschwartz - why don't you let the OP speak for himself? Did he request that you monitor his threads and speak for him?

BR

[eschwartz]

Originally Posted by BetterRed

@eschwartz - why don't you let the OP speak for himself? Did he request that you monitor his threads and speak for him?

BR

No, but sometimes I have opinions without getting an engraved invitation first. No need to monitor anyone's threads either, I just speak up whenever I stumble across things.

And the OP has already spoken for himself. I pointed that out.

I then proceeded to have an opinion in the same manner in which Purple Lady, chaley, and theducks had an opinion, in which I concluded that their concerns are not worth bothering about for a redundant backup when Add books can restore everything. Emphasis on using it as a redundant backup, emphasis on the part where I stated a workaround.

I assume I am also allowed to have opinions?

[BetterRed]

Ah but I do not see where chaley, Purple Lady, theducks et al say anything like "The OP is well aware of..."

By all means voice your own opinions and knowledge, but in your own name.

IMO without their express wish that you do so. its actually disrespectful to speak on behalf of others. Unless they're being bullied, attacked or provoked which is clearly not the case here

I guess they don't they teach manners or etiquette these days - Rafferty's Rules rulz.

BR

[cybmole]

OP here.

I have previously read much discussion of google drive issues.
I understand that it corrupts calibre operation when used as a live library.
inspecting my reduntant google drive backup, all the numerics in folder names look OK.
if I were to restore from that, then i'd do 2 steps. 1, copy all to a local folder. 2. then and only then tell calibre to use that local folder as an existing library.
if that did not work then , as has been pointed out, I could re-import every book to a new library.

maybe we need a definitive sticky on the issues with cloud storage; it would appear from recent posts that we don't have a consensus yet on exactly when/ how goolge drive corrupts things so maybe we also need more objective test results.

bear in mind that google drive is not an actual local copy of a cloud - it's not like a sugar sync or dropbox folder . its a "fake" folder whose contents exist only in the cloud. ( there is an offline mode but it is off by default )

let me start that ball rolling. I have side by side, my local calibre library & my google drive copy. I am opening each author folder in both locations & checking the 4 digit numbers. I checked the 1st 3 authors & get 3 out of 3 matches. so I conclude no corruption, which is what I expected.

[chaley]

Originally Posted by cybmole

bear in mind that google drive is not an actual local copy of a cloud - it's not like a sugar sync or dropbox folder . its a "fake" folder whose contents exist only in the cloud. ( there is an offline mode but it is off by default )

It doesn't work like this for me. I have changed no option and with the exception of google docs files, all my data is stored locally.

let me start that ball rolling. I have side by side, my local calibre library & my google drive copy. I am opening each author folder in both locations & checking the 4 digit numbers. I checked the 1st 3 authors & get 3 out of 3 matches. so I conclude no corruption, which is what I expected.

Concluding no corruption on three samples is a bit like me concluding that all people in Paris are women (except me) because the last three people that went by my window were women.

I have personally had Google Drive change those numbers. It happened when there were two machines syncing the drive, but the library with the numbers was written by only one machine. I don't know why it did it at that time, but it was indisputable that a four digit number became a single digit. I no longer let my calibre library anywhere near GD for any purpose.

My position: a backup that might change behind your back, even though it apparently hasn't yet, is not a backup. It might be considered an archive, useable as eschwartz says to reconstruct a library by reimporting, but that is not a backup.

[BetterRed]

@cybmole - I doubt you'll get a consensus on google-drive.

If one creates a Sticky saying its OK to use GD this way but not that way, but its OK to use OD or DB either way then there is the danger that someone will misinterpret. And then there are other services like Copy.com, and the next cab off the rank, are they OK to use and if so how should they be used.

I vote leave things as they are - with the Sticky that DoctorOhh created in Devices, and the warning in FAQ.

Unless of course, someone can demonstrate that the previously reported issues with GD have been fixed. And the folder names never get clobbered no matter what - ie it behaves like Dropbox, OneDrive, Copy.com etc. Then perhaps the Sticky can be removed and the FAQ can be updated.

Speaking for myself whilst ever the warnings remain in place I will suggest that GD be avoided, same as I do for putting a library on a network drive. Someone will always push the edge of the envelope and then come back saying... but you said....

BR