Library management issues after editing metadata

[dra]

Hi.

No amount of search aided me in finding a solution to this. With caution that my searching might have been faulty, please allow me to describe the very basic issue I am facing:

1. I edit the metadata of a record in Calibre. For example, the author's name is changed, or the book's title. This record contains an actual book in any of the formats acceptable (pdf, epub, etc) and an actual image cover.
2. I click OK.
3. The record now contains the following: the generic cover as image that Calibre has by default, and a book in the same file format before clicking OK but now containing nothing. It cannot even be opened.
4. The file path before clicking OK and the file path after I click OK are different. Calibre did not rename the file and did not take the file with it's cover to the new file path. It simply created a new folder, inserted the generic cover and a new blank book. It would seem that by browsing the library the book has been corrupted or lost, but in reality it's in the directory.

How do I tell Calibre to rename the file name and take it together with it's cover to the new folder it creates?

Your help is very much appreciated!

[kovidgoyal]

Put your calibre folder on a local hardisk running a filesystem native to whatere operating system you are on, and one that is not mirrored by any cloud service and you will be fine.

https://manual.calibre-ebook.com/faq...rked-drive-nas

[dra]

Originally Posted by kovidgoyal

Put your calibre folder on a local hardisk running a filesystem native to whatere operating system you are on, and one that is not mirrored by any cloud service and you will be fine.

https://manual.calibre-ebook.com/faq...rked-drive-nas

Thanks for that.

However, I have my doubts.

The issue described is being met on a PC running windows. The library is saved on drive file stream (g suite).

I am not facing this issue on the mac, which has exactly the same setup (library saved on drive file stream).

It would seem to me that if the issue is what you say, it would not work on a mac; but it does.

Any other ideas?

[stumped]

"The library is saved on drive file stream (g suite)."
Thats the problem, right there
What part of use only a local drive in the FAQ did you not understand?

G suite is the business version of Google drive with all the same calibre incompatibilities and issues
Search this forum for references to Google drive for the full horror story

And it will go wrong on Mac also, at some point. You have just been lucky so far.

[dra]

Originally Posted by stumped

"The library is saved on drive file stream (g suite)."
Thats the problem, right there
What part of use only a local drive in the FAQ did you not understand?

G suite is the business version of Google drive with all the same calibre incompatibilities and issues
Search this forum for references to Google drive for the full horror story

And it will go wrong on Mac also, at some point. You have just been lucky so far.

There's no part I did not understand, but thanks for the passive aggressiveness.

I am quite aware what g suite's google drive file stream is, again thanks for letting me know something already clear in my earlier comment. I never doubted it is cloud storage; anyone reading my comment, having half an ounce of brain, can understand that.

Not going wrong on a mac until 'some point' being explained by 'luck' doesn't explain much.

After you're done with the attitude, here it goes again:

**I have my doubts since it works perfectly fine on the Mac with the exact same setup**

When something is an absolute, it doesn't have 'luck' working around the absolute. The original answer is stating an absolute; paraphrasing it: 'you cannot use a cloud storage service as library storage because it's the reason the problem is being created'. However, I do not meet that absolute in my experience using calibre on the mac with drive file stream as the library storage.

**Therefore** I am asking about any other possible explanation.

If you don't have an alternative one, then you're of no help to this issue.

Using a keyboard to burst out other issues you may have, of self-appreciation or lack thereof, again helps in no way, shape or form in answering the question.

I'm sure you're a big joy having around in any case, just not this one.

[chaley]

Originally Posted by dra

There's no part I did not understand, but thanks for the passive aggressiveness.

...

Regardless, stumped is correct. The fact that it "works" on the mac is proves nothing.

The fact is that calibre *will fail* in the way you describe if you use directories/folders that are syncronized with something such as the cloud, or that are stored on a networked device. It may fail immediately or it may fail intermittantly, but it *will fail* eventually. Believe me or don't. Your choice.

[dra]

OK, for sake of conversation I accept that at some point it will go t*ts up.

In that case, let's then go on to the main issue: how do we setup a cloud backup of the calibre library?

[theducks]

Timing is everything (to Calibre). It take little to flip to the Not Working side.

To be less susceptible to those changes, you would need to freeze all aspects of the connection. Obviously you can't (and should not) forgo updates to security and OS Patches.
Add in the Covid 19 effects on Network Traffic, and I am really surprised that the Internet has been as stable as it has.
So, IMHO, Luck has been with you till now

[gbm]

Originally Posted by dra

OK, for sake of conversation I accept that at some point it will go t*ts up.

In that case, let's then go on to the main issue: how do we setup a cloud backup of the calibre library?

https://manual.calibre-ebook.com/faq...rked-drive-nas

Do not put your calibre library on a networked drive.

A filesystem is a complex beast. Most network filesystems lack various filesystem features that calibre uses. Some don’t support file locking, some don’t support hardlinking, some are just flaky. Additionally, calibre is a single user application, if you accidentally run two copies of calibre on the same networked library, bad things will happen. Finally, different OSes impose different limitations on filesystems, so if you share your networked drive across OSes, once again, bad things will happen.

Even with these tools there is danger of data corruption/loss, so only do this if you are willing to live with that risk. In particular, be aware that Google Drive is incompatible with calibre, if you put your calibre library in Google Drive, you will suffer data loss

bernie

[BetterRed]

Originally Posted by dra

In that case, let's then go on to the main issue: how do we setup a cloud backup of the calibre library?

What many people do is use a folder/file tool like rsynch, free file sync, etc to mirror their libraries to a local USB HDD, cloud storage, etc.

I use Goodsync to mirror all the data I'd prefer not to lose to a local USB HDD daily, and to an FTP server weekly.

There's a swag of threads here on calibre library backups, do a search for backup in this sub-forum.

BR

[stumped]

Originally Posted by dra

OK, for sake of conversation I accept that at some point it will go t*ts up.

how gracious of you to concede that the program author, and several other experts, all saying the same thing , could hypothetically be right.

In that case, let's then go on to the main issue: how do we setup a cloud backup of the calibre library?

very easily, . Go search.

[theducks]

free file sync is amazingly fast.
I had been using Synctoy, but it would not run on my new Dell because it wanted an older dot net than is available from MS. FFS is ~10x faster.

Setting a sync pair is pretty straight forward. Choose the source folder, choose the Target. Set the sync to MIRROR (2-way will corrupt the Library over time)

[stumped]

Originally Posted by theducks

free file sync is amazingly fast.
I had been using Synctoy, but it would not run on my new Dell because it wanted an older dot net than is available from MS. FFS is ~10x faster.

Setting a sync pair is pretty straight forward. Choose the source folder, choose the Target. Set the sync to MIRROR (2-way will corrupt the Library over time)

The thing about cloud corruption is that it can be very gradual/subtle. you go along thinking all is good and that your 10,000 calibre files are all there, in the correctly named subfolders, until one day a book format is missing or invalid. Then you open the bonnet and run checks, but you are outside the 15 / 30 day rollbacks that the free cloud plans provide....

its not just a calibre thing. I remember when I was trying to upload 10+ years of family photos to google and to amazon clouds.
I had the official sync tools and was doing them in batches, but about 1 per 100 would randomly fail to upload. Ok if you spotted it, and it would upload just fine as a single manual upload. The moral is than you can't /shouldn't 100% trust the tools they give you. both amazon and google products were awful actually, left to their own devices they were constantly starting over and re syncing previous uploads. eventually I dropped the whole idea of letting the amazon/ google desktop sync products run things by keeping a local drive folder in permanent sync. I reverted to : open a browser for the cloud drive , drop in a sub folder, check for errors, rinse, repeat

Also, I had many many Gbs of photos all organised into yy-mm sub folders- and I could see that structure reflected in google drive ( taking advantage of the unlimited photos storage deal)
Sometime later google did one of their (in)famous overhauls, threw google drive and google photos into a blender, and all the photos went into one ginormous photos folder, managed by tags, not by folder names. Maybe they are all in there,somewhere, but the option to download a specific yymm subfolder back to desktop has vanished.


tl:dr don't trust cloud products to provide long term storage

I trust dropbox more than most, but I stick to the free tiers. Because one accidental failure to pay the paid plan monthly bill ( because your card expires or your bank has a glitch) could result in a mass delete.

[Sirtel]

Originally Posted by stumped

I trust dropbox more than most, but I stick to the free tiers. Because one accidental failure to pay the paid plan monthly bill ( because your card expires or your bank has a glitch) could result in a mass delete.

They don't delete all your stuff right away. They send you a notice first. Possibly several notices, and they give you a deadline. If it was that simple (one missed payment = immediate mass delete), then no one would use their paid plans, because accidents happen with payments.

[stumped]

sure, but accidents can still happen. the notices go to some email you used at the time, but have moved on, so it's defunct or you don't check it... or your bank gets over-zealous about online fraud and declines a payment...
e.g. my 15Gb dropbox email ( built up with lots of referrals) is years obsolete, but I can't change it to my current email as I already have a different small free dropbox on that.
if you read trustpilot reviews of cloud services, you will see many unfortunates reporting that all their data was wiped after payment hassles.

[ the declined payment is a genuine problem now in UK. Covid came along just as banks were upping online card payment security, with texts, PINs, that have to be responded to before payment is allowed. I have read actual horror stories from self-isolating folks ordering groceries on line- they put in a order, the online retailer goes for payment authorization not until after midnight the day before, just in time for the am delivery. Bank sends a text to customer saying please confirm the payment, but customer is sound asleep by then. Payment fails, delivery is cancelled, and the next available delivery slot is 3 weeks away if you are lucky... ]

a more mundane example _ i was sharing my son in law's google play music family plan. it suddenly stopped working and started asking me to subscribe. I thought I must have upset him & gotten ditched... turns out there was some glitch with his credit card number changing or expiring and google had instantly pulled the subscription until it was sorted. no grace period, no benefit of the doubt...