I have finally started work on adding a "cloud connection" to CC. In the process of prototyping such a thing I have run into some issues that raise questions. I am posting this in hopes of getting some information that will guide me as I "finish" this feature.
Some basics:
- I decided not to try to monitize it. The cloud connection will arrive in CC for anyone and everyone.
- The connection will appear in the same way as the existing content server. The first page will show you more-or-less what the top content server page shows. and so on.
- "Downloading" a book will add it to CC's library.
- I assume that providing more cloud providers than Dropbox would be a good thing.
Now to the questions:
- The cloud APIs are a mess. Example: if I choose to use the Dropbox API then I am more-or-less stuck with it, The impact: developing a more generic API will require the user to authenticate using the web browser and not the local app, meaning you will be required from time-to-time to sign in to your provider on a web page. "Time-to-time" might be as often as every connection. Is this OK? Note that if the answer is "no", the result will be that CC will support an extremely limited number of cloud providers, specifically those who provide a software library. Currently that is Dropbox, perhaps Box, and even less perhaps Onedrive. Copy.com is not on the list. I am refusing to support Google Drive because it can destroy a calibre library.
- The cloud providers are *extremely slow". For example, it can take 5 to 10 seconds to download a cover from Dropbox. Should we even try to cache covers when browsing categories? Is consuming the space to store the cached covers worth it?
- There seems to be no way to use the account information stored on the device. This means that when you connect to the cloud you might be required to log in. Dropbox does not suffer from this problem because it gives back a token that never expires. The other providers provide login tokens that expire after a time. Is the benefit of allowing other providers worth the user hassle of having to re-login on (I think) each connection? Note that I cannot ask for and store credentials (usernames and passwords) because the underlying programming interfaces do not provide a way for an app to log itself in.
- Searching is extraordinarily slow, 3 to 5 books per second. Is searching really required, given that you will be able to navigate the item hierarchy just as you can in the current content server interface?
- Would it be useful for this connector to access a complete calibre library downloaded to the device? CC would copy desired books from this library copy into its database. It would be the responsibility of the cloud provider/program to keep the calibre library copy up-to-date. It seems to me to be a bit strange that the entire library is copied to the device but not to CC.
On the good side, once the calibre metadata.db is downloaded the interface is very fast (ignoring covers). That raises a question: would it be useful for CC to accept manually-downloaded databases from "unusual" cloud providers? Going this route would require the user manually to download a copy of calibre's db, then later to manually download the desired books after CC gives you the file path information. Cover thumbnails would not appear except (perhaps) in book details and then only after the user intervened and downloaded the cover jpg. My feeling is that this is too much of a pain in the ^% for users to justify the development.
EDIT: some updates:
- The cover slowness problem has been resolved. It now takes around 1.5 seconds per cover. If more than one cover is needed they are fetched in parallel. If one cover takes 1 second, 10 covers takes around 2 seconds.
- I have now using the base-level APIs so supporting multiple cloud providers is possible. I have working implementations for Dropbox and Onedrive.
- For the two implemented providers, authentication is a one-time thing. It happens the first time you connect to a given provider.