MobileRead Forums - View Single Post

chaley · 08-14-2014, 01:04 PM

I have been thinking for some time about whether and how CC might work with calibre libraries stored in the cloud, such as in Dropbox. As far as I can tell, there are three possible directions:

Connect to the library in the cloud via its local app. This means that the calibre db must be downloaded to the device, as well as all the covers. It opens the issue of detecting obsolescence, and will have "interesting" performance characteristics.
Add a function to CC that can import a book from a calibre-style folder on the device, which is one that contains formats, metadata.opf, and cover.jpg. The disadvantages are that the folder must somehow get to the device, and that the metadata.opf file in the folder must be up-to-date.
Connect to a local calibre library downloaded from the cloud to the device. This has the advantage of performance, because everything is local. It solves the obsolescence problem, at least if the underlying cloud sync works. It has the disadvantage of space; all the book formats are stored locally.

Calibre Cloud free does #1. Calibre Cloud Pro does #1 and #3, as does Leger Calibre and perhaps others. No one does #2 as far as I know.

My problems with #1 are (in no particular order):

By necessity it would need to be a part of CC, because downloaded books need to be managed locally.
It would be slow, certainly no faster than Calibre Cloud.
It would require us understanding the API of several cloud storage providers.
It would be difficult to monetize. It is a lot of work both to build and maintain, and we need to get something out of such a significant development effort.

Number 3 doesn't have any of the above problems. There is no need at all for it to be integrated into the current CC, permitting it to be a separate app (lets call it CClocal), chargeable separately. CCLocal's performance would be similar to CC, in some cases faster and in some cases slower. There is no need to understand any API, because the user would choose the synchronization method. CCLocal should have the same functionality as the current CC, with the exception of the "Connect" operations. The grouping drawer, grid + list views, book details, settings, etc would work as now, with the exception of "columns built from other columns" (see below). One exception to functional equivalence might be the read/date read information. It isn't clear how to sync this information with calibre. We cannot change the DB because that would certainly create conflicted copies.

Number 2 has the advantage of simplicity from CC's point of view. A folder appears on the device by magic and CC imports that folder. The downside is that there is no assistance to find the folder to download. The user must find that folder manually then download it.

Both #1 and #3 will suffer from not being able to use values in "columns built from other columns". The value is not stored in calibre's database and there is no way that CC or CCLocal will implement calibre's template language processor. They could in theory get the values from the metadata.opf files in the book folders, but doing so would be glacially slow.

An argument against doing CCLocal is that it is difficult to see any significant advantage offered by CCLocal over simply sending one's entire library to the current CC. And of course, sending one's entire library obviates the need to do any of #1, #2, and #3.

It seems to me that #3 is superior to #1. The drawbacks are the space required to hold a copy of calibre's library on the device and the possibility of losing the ability to set is_read info, and I am not convinced that these are enough of a problem to force #1 over #3.

I wonder about #2. It would be rather straightforward to build into CC, removing the need to monetize it. It would be part of CC, thus supporting is_read and the like. Calibre tries hard to keep the metadata.opf files up-to-date. The values for columns built from other columns are available in the metadata.opf file. Of course the major problem, and potentially a killer problem, is the requirement to find a book by navigating the calibre library folder structure and downloading the desired folder.

Comments? When commenting, please assume you get to choose only one option. For me, "Do them all" is equivalent to "Do none of the above".

08-14-2014, 01:04 PM	#1
chaley Grand Sorcerer Posts: 12,467 Karma: 8025600 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	CC and the cloud: discussion I have been thinking for some time about whether and how CC might work with calibre libraries stored in the cloud, such as in Dropbox. As far as I can tell, there are three possible directions: Connect to the library in the cloud via its local app. This means that the calibre db must be downloaded to the device, as well as all the covers. It opens the issue of detecting obsolescence, and will have "interesting" performance characteristics. Add a function to CC that can import a book from a calibre-style folder on the device, which is one that contains formats, metadata.opf, and cover.jpg. The disadvantages are that the folder must somehow get to the device, and that the metadata.opf file in the folder must be up-to-date. Connect to a local calibre library downloaded from the cloud to the device. This has the advantage of performance, because everything is local. It solves the obsolescence problem, at least if the underlying cloud sync works. It has the disadvantage of space; all the book formats are stored locally. Calibre Cloud free does #1. Calibre Cloud Pro does #1 and #3, as does Leger Calibre and perhaps others. No one does #2 as far as I know. My problems with #1 are (in no particular order): By necessity it would need to be a part of CC, because downloaded books need to be managed locally. It would be slow, certainly no faster than Calibre Cloud. It would require us understanding the API of several cloud storage providers. It would be difficult to monetize. It is a lot of work both to build and maintain, and we need to get something out of such a significant development effort. Number 3 doesn't have any of the above problems. There is no need at all for it to be integrated into the current CC, permitting it to be a separate app (lets call it CClocal), chargeable separately. CCLocal's performance would be similar to CC, in some cases faster and in some cases slower. There is no need to understand any API, because the user would choose the synchronization method. CCLocal should have the same functionality as the current CC, with the exception of the "Connect" operations. The grouping drawer, grid + list views, book details, settings, etc would work as now, with the exception of "columns built from other columns" (see below). One exception to functional equivalence might be the read/date read information. It isn't clear how to sync this information with calibre. We cannot change the DB because that would certainly create conflicted copies. Number 2 has the advantage of simplicity from CC's point of view. A folder appears on the device by magic and CC imports that folder. The downside is that there is no assistance to find the folder to download. The user must find that folder manually then download it. Both #1 and #3 will suffer from not being able to use values in "columns built from other columns". The value is not stored in calibre's database and there is no way that CC or CCLocal will implement calibre's template language processor. They could in theory get the values from the metadata.opf files in the book folders, but doing so would be glacially slow. An argument against doing CCLocal is that it is difficult to see any significant advantage offered by CCLocal over simply sending one's entire library to the current CC. And of course, sending one's entire library obviates the need to do any of #1, #2, and #3. It seems to me that #3 is superior to #1. The drawbacks are the space required to hold a copy of calibre's library on the device and the possibility of losing the ability to set is_read info, and I am not convinced that these are enough of a problem to force #1 over #3. I wonder about #2. It would be rather straightforward to build into CC, removing the need to monetize it. It would be part of CC, thus supporting is_read and the like. Calibre tries hard to keep the metadata.opf files up-to-date. The values for columns built from other columns are available in the metadata.opf file. Of course the major problem, and potentially a killer problem, is the requirement to find a book by navigating the calibre library folder structure and downloading the desired folder. Comments? When commenting, please assume you get to choose only one option. For me, "Do them all" is equivalent to "Do none of the above". Last edited by chaley; 08-15-2014 at 05:06 AM.