What you're doing should be possible:
https://www.youtube.com/shorts/4Z_w4CeTeQI
I installed calibre companion on my macbook to test it out, but it looks like there's a subscription so I don't think I'd use that, but it looks like it should work as long as you have your calibre library in onedrive properly, i.e. a folder "Library" that contains the folders for authors and the database file, and you select the Library folder to open.
If the error you keep getting is a that there is no existing library, it might be that the database file is missing, possibly? I think that's what allows the program to identify that there is a library there.
I have never personally tried to do it this way, I used calibre-web once and I liked that— as long as I was on my local network I could access the server run from my laptop. But I just am so used to Apple Books that I transfer all the epub files into itunes... it's awful but I can't stand the interface of other readers.
Please provide more details of your calibre library in one-drive if it still didn't work.