Issue with OneDrive update (macOS)

[tomsem]

Somewhere in last few months, Microsoft has started naming the 'real' folder name to OneDrive-Personal (I guess to distinguish it from OneDrive Enterprise). It used to be just /OneDrive.


In Finder, one still sees 'OneDrive' in Locations, and if you Go To Folder /Users/username/Library/CloudStorage/. So it's some sort of Finder alias.

Full path name on macOS is:

/Users/[accountname]/Library/CloudStorage/OneDrive-Personal

It still shows the folder name as 'OneDrive' in Finder Locations.

When starting calibre, I was getting 'cannot find catalog' error, which proposed creating a new catalog, which I did not want to do. So I was quitting calibre, and somehow stumbled on gui.json, where I discovered two entries (I think order was reversed):

  "library_usage_stats": {
    "/Users/thomas/Library/CloudStorage/OneDrive-Personal/calibre": 19,
    "/Users/thomas/Library/CloudStorage/OneDrive/calibre": 384
  },

"Obviously," I told myself, "that OneDrive-Personal path is wrong". I removed it, calibre launched without error. But only until the next launch.

Then I discovered that cancelling the error dialog opened the existing location and I was able to just click Open. But continued to present error on each launch.

Finally googled OneDrive-Personal and realized I had been removing the wrong entry. I removed the (incorrect) '/OneDrive/', and since doing this, calibre has been launching without error.

However, the (incorrect) '/OneDrive/' entry is getting restored for some reason. It's apparently not causing problems because evidently calibre is able to update library usage stat for the correct location without tripping over it. But JSON doesn't define 'order' and so json package would not be wrong to read it in to Python dict in the other order, and then error will recur.

Is there some way to keep the incorrect entry from getting restored? I'm looking at files in /Users/thomas/Library/Preferences/calibre and don't see any other instance of '/OneDrive/' string in any of the files there.

[Sirtel]

You should not keep your active library in any folder OneDrive is syncing. Only a mirrored copy.

[tomsem]

Originally Posted by Sirtel

You should not keep your active library in any folder OneDrive is syncing. Only a mirrored copy.

I want it backed up. I've done this for years without any issue, until this one, which is not related to syncing. I'm only updating catalog from one computer.

[Sirtel]

Originally Posted by tomsem

I want it backed up. I've done this for years without any issue, until this one, which is not related to syncing. I'm only updating catalog from one computer.

As I said, you can mirror a copy to OneDrive. The thing with an active library on a network drive or a cloud drive is that it works until it doesn't, and by the time you notice the problem it may already be too late to fix, unless you also have a backup somewhere else. This is from personal experience (had a bad data loss with OneDrive years ago, fortunately I had a back-up elsewhere).

[kovidgoyal]

library_usage_stats is not used to choose the startup library unless connecting to the existing one fails.

[tomsem]

Originally Posted by kovidgoyal

library_usage_stats is not used to choose the startup library unless connecting to the existing one fails.

Okay, so where does it store path for 'existing startup library'?

[kovidgoyal]

If you grep for the path you will find it in gloabl.py.json

[tomsem]

Originally Posted by kovidgoyal

If you grep for the path you will find it in gloabl.py.json

Thank you! (assume it is global.py.json?)

[tomsem]

Originally Posted by Sirtel

As I said, you can mirror a copy to OneDrive.

I seem to have web search dyslexia. I cannot find anything that explains how to do it on macOS.

How are you doing this?

[Sirtel]

Originally Posted by tomsem

I seem to have web search dyslexia. I cannot find anything that explains how to do it on macOS.

How are you doing this?

I use FreeFileSync (mirror option, not two-way). After the initial copy has been created, it only updates the files that have been changed, so it's very fast. There is a MacOS version too.

[tomsem]

Originally Posted by Sirtel

I use FreeFileSync (mirror option, not two-way). After the initial copy has been created, it only updates the files that have been changed, so it's very fast. There is a MacOS version too.

Thanks, I'll check it out.

[tomsem]

I had the error recur and ran calibre-debug -g to see what might have been the trigger.

Saw this:

Traceback (most recent call last):
File "calibre/gui2/main.py", line 331, in initialize_db
File "calibre/db/legacy.py", line 195, in __init__
File "calibre/db/legacy.py", line 50, in create_backend
File "calibre/db/backend.py", line 507, in __init__
File "calibre/db/backend.py", line 2011, in ensure_trash_dir
File "calibre/db/backend.py", line 2035, in expire_old_trash
File "calibre/db/backend.py", line 413, in rmtree_with_retry
File "shutil.py", line 717, in rmtree
File "shutil.py", line 630, in _rmtree_safe_fd
File "shutil.py", line 626, in _rmtree_safe_fd
NotADirectoryError: [Errno 20] Not a directory: '/Users/thomas/Library/CloudStorage/OneDrive-Personal/calibre/.caltrash/b/.DS_Store'

.DS_Store is a Finder artifact, no idea how it could have wound up in there (there was another in .caltrash/), or even if it could cause the issue I'm seeing. But it's gone now, and so far, so good.

The library_usage_stat turd has remained 'gone'. Apparently it's created when one select the library folder with system dialog, where you only see 'OneDrive'. Is Qt not returning real path or something?

[kovidgoyal]

That error indicates something on your system is creating/deleteing/renaming files inside the folder while calibre is trying to delete it. Having multiple things write to the same locations on the file system is a recipe for data loss.

[tomsem]

Perhaps, though it is not at all obvious what the 'something' might be.

In any case, everything is working as expected now.

Thanks for your help.