Calibre [Errno 22] with Onedrive "File on demand" activate / General sync suggestion

[tweecore]

Hello everyone,

This is my first post here, although I have often received help from threads on this forum in the past up until today, for which I want to thank all users. So this seems the right place where to seek for help.

I've been a Calibre user for the last 10 years, during which I have collected and painstakingly catalogued 7500+ books.
Recently I purchased an Android tablet, with the intention of accessing my Calibre library from there as well.
You can call this the general long-term objective, so please feel free to suggest other, better solutions than the one I'm trying.
Second in priority, but ideal, would be to pursue what I'm already trying to do: to have Calibre function as before while my library is in the cloud, not locally stored on my laptop – that is, accessible through OneDrive's "File On Demand" option. Since my library is large in size, I thought of going for a cloud storage service not to fill up my hard drive in the first place.

So I finally decided it was time to get a OneDrive subscription and move all my library to the cloud. I followed some guidelines and followed them as much as I could. The only problem is that in the many threads I read on this forum and elsewhere, I never managed to understand if what people was trying to do was the same, i.e. having the library as "File on demand", OR having the library locally stored and on the cloud as well.
These are the steps I took:
- I moved my Calibre library to the default OneDrive directory (C:\User\*\OneDrive\)
- I quit Calibre and ran OneDrive to sync all the files to the cloud (with the exception of the metadata.db and backup, that I kept both locally stored as well)
- When OneSync has finished, I opened Calibre. This led to an overlap of, like 5 seconds, when Calibre and OneDrive where running simultaneously (which I understand to be a big no-no), and a very scary message from OneDrive trying to sync "calibre.exe" (idiot me )
- Now, Calibre loads my library, I had no loss of metadata: all my books are there. Except that, perhaps logically, I can click only on those files that are locally stored. So if I need one book, I download it locally from my OneDrive, and it is properly displayed and accessible on Calibre. If on Calibre I select any other cloud-only file, it gives me this message:

calibre, version 5.9.0
ERRORE: Eccezione non gestita: <b>OSError</b>:[Errno 22] Invalid argument

calibre 5.9  embedded-python: True is64bit: False
Windows-10-10.0.18362 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.18362')
Python 3.8.5
Windows: ('10', '10.0.18362', '', 'Multiprocessor Free')
Interface language: it
Successfully initialized third party plugins: Quality Check (1, 11, 0)
Traceback (most recent call last):
  File "calibre\gui2\library\models.py", line 571, in current_changed
  File "calibre\gui2\library\models.py", line 556, in get_book_display_info
  File "calibre\gui2\library\models.py", line 734, in cover
  File "calibre\db\legacy.py", line 448, in cover
  File "calibre\db\cache.py", line 723, in cover
  File "calibre\db\cache.py", line 72, in call_func_with_lock
  File "calibre\db\cache.py", line 763, in copy_cover_to
  File "calibre\db\backend.py", line 1445, in copy_cover_to
  File "shutil.py", line 202, in copyfileobj
OSError: [Errno 22] Invalid argument

Now, this (Calibre not being ablet to load files that it can't find on the local drive) feels pretty logical to me. However, from what I've read around, there seems to be plenty of people using OneDrive and Calibre.
Are they all storing their library locally, and have it ALSO on the cloud?

• Is there a way to have my library on a "File on demand" basis, and have Calibre let me select any book without the error? (I understand that, if I wanted to edit the metadata of a book, I would have to download the corresponding folder locally to do so, but now I can't even click on a book...)
• In more general terms, what is the best way to have my library in the cloud and have Calibre operate as before (when I stored my library locally), if there is one?
• Even more generally, what is the best configuration you would suggest to achieve what I'm generally trying to achieve – namely, acessing my books from my tablet in a way that's synchronised with the library I access and manage from my laptop?

I know I'm asking a lot, but I also feel that there might be simple answers to what I'm asking. And sorry for being overly long; I feel the more info you have, the more you might be able to help me.
This being my first post, please pardon me for inexperience. And sorry in advance to the admins if I'm posting this in the wrong section.

Thanks to everyone that might help me!

MY SPECS:

DELL XPS 13 7390
Windows 10 Pro, v1909
Calibre 5.9.0

[tweecore]

Aaand now Calibre doesn't even open anymore
When I runit, the launch window opens up, stops and "Initializing graphic interface" than crashes. Tried to uninstall and reinstall with no success. HELP!

[kovidgoyal]

https://manual.calibre-ebook.com/faq...rked-drive-nas

[tweecore]

Kovid,

First of all let me thank you for creating this incredible software and be so much an active part of its community that you take your time to answer me.
I did read what you're linking me, and tried to do it all the same. I shouldn't have, but it looked like to me that many other users were doing the same without much issues. As the FAQ says to use a file sync tool like Dropbox instead of a networked drive, I thought I was good enough with OneDrive.
Then I'm afraid I f***ed up when I let Calibre and OneDrive run symultaneously before realizing it (really a question of seconds), as the FAQ specifically says it is essential that this must not happen.

What can I do now to be able to run Calibre again? I can of course download back all my library to the local drive, but will Calibre be able to launch again? As I said in the previous post, now it crashes at startup. How can I fix this, before I can go back to focus on the issues that I had when start the thread?

[kovidgoyal]

simply delete your current library folder (assuming you have a backup) and calibre should start fine. Once it has started, copy your backup somewhere local and point calibre to it by clicking the calibre icon in the main window calibre toolbar and choosing switc to library.

[Sirtel]

As to other people keeping their library on a cloud drive successfully - I can only speak for myself, but yes, my library in Dropbox is locally stored on all my devices, as well as in the cloud. I never use the Dropbox counterpart of "File on Demand" for my Calibre library.

As my library is over 60GB by now and my whole Dropbox folder over 300GB, I have to choose my laptops and tablets accordingly. The average 256GB of storage just won't cut it.

[gtriever]

OT: This is why I'm a cloud-hating Luddite. Hard drives are so inexpensive now that I just can't see myself depending on a cloud-based "whatever". My Library resides on a 4TB RAID-1 internal HDD set, and that's backed up to (2) separate 4TB USB Hard Drives.

Good luck with getting your library restored and working again. Calibre is a wonderful piece of software, and Kovid's been great with his support.

[tweecore]

Originally Posted by kovidgoyal

simply delete your current library folder (assuming you have a backup) and calibre should start fine. Once it has started, copy your backup somewhere local and point calibre to it by clicking the calibre icon in the main window calibre toolbar and choosing switc to library.

I tried to rename of the Calibre directory folder to see what would have happened, and Calibre opened just fine, but obviously with an emptuy library. Now I am downloading back all my files to a local folder as you suggest, and will point Calibre to the new location. Thank you so much for your quick and kind help!

Originally Posted by Sirtel

As to other people keeping their library on a cloud drive successfully - I can only speak for myself, but yes, my library in Dropbox is locally stored on all my devices, as well as in the cloud. I never use the Dropbox counterpart of "File on Demand" for my Calibre library.

As my library is over 60GB by now and my whole Dropbox folder over 300GB, I have to choose my laptops and tablets accordingly. The average 256GB of storage just won't cut it.

Thank you for your advice, Sirtel. I will keep my library stored locally (and cloud) from now on. What would you suggest then, in order to access the files from other devices?
The options I have in mind are:
- I put my library on an SD card, manage it from Calibre (laptop), and put it in the tablet when I need something. But I wouldn't have the comfort of a Calibre-like catalogue on the tablet – wouldn't I?
- Put my library on an SD card, manage it from Calibre (laptop), and choose the SD card as the location of my OneDrive directory. This way I would have the file stored locally AND on the cloud, while occupying no space in the laptop SSD.
- I store my library somewhere local, and have OneDrive do planned backups (if it has such a feature) of that directory. Then access the backup in the cloud from the tablet. But it sounds a bit convoluted.

In other words, I'm looking for the best solution in order not to have my library on my laptop's SSD, and be able to share it with my tablet in a smooth way.

[tweecore]

Originally Posted by gtriever

OT: This is why I'm a cloud-hating Luddite. Hard drives are so inexpensive now that I just can't see myself depending on a cloud-based "whatever". My Library resides on a 4TB RAID-1 internal HDD set, and that's backed up to (2) separate 4TB USB Hard Drives.

Good luck with getting your library restored and working again. Calibre is a wonderful piece of software, and Kovid's been great with his support.

I was totally like you gtriever until a while ago. The turning point for me was realizing I have no idea how many external HDDs I have at this point, SD cards and so on. I'm a compulsive archiver, and not only of books. Turning to a cloud service sounded like a good idea now that I wanted to be able to sync across devices... But now I see the many issues.

[Sirtel]

Originally Posted by gtriever

OT: This is why I'm a cloud-hating Luddite. Hard drives are so inexpensive now that I just can't see myself depending on a cloud-based "whatever". My Library resides on a 4TB RAID-1 internal HDD set, and that's backed up to (2) separate 4TB USB Hard Drives.

I don't depend on the cloud, I use it for convenience. All my files are backed up to several external drives as well. But I want to use my files on different devices (at least 2 laptops and a Windows tablet) and nothing beats Dropbox in this. I can't imagine lugging around external drives every time I want to access something, especially if I'm not at home.

[thiago.eec]

Originally Posted by tweecore

What would you suggest then, in order to access the files from other devices?

Have you tried calibre server? You can view, manage and read your books, using a web browser on other devices, like phones, tablets and computers.

[Sirtel]

Originally Posted by tweecore

I was totally like you gtriever until a while ago. The turning point for me was realizing I have no idea how many external HDDs I have at this point, SD cards and so on. I'm a compulsive archiver, and not only of books. Turning to a cloud service sounded like a good idea now that I wanted to be able to sync across devices... But now I see the many issues.

For an Android tablet, I suggest using the Calibre Companion app. You'll have your library structure without the actual files taking up space on the device.

[tweecore]

Originally Posted by Sirtel

For an Android tablet, I suggest using the Calibre Companion app. You'll have your library structure without the actual files taking up space on the device.

I've tried CC before doing the cloud thing, but the app doesn't let me connect to Calibre because I'm on a Public (student housing) network: can't change IP configuration, open ports, none of the sorts...