View Single Post
Old 03-14-2016, 04:41 AM   #7
chaley
Grumpy old git
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
chaley's Avatar
 
Posts: 9,927
Karma: 3493677
Join Date: Jan 2010
Location: UK
Device: Reader
I may seem like I am ignoring you. I'm not.
Quote:
Originally Posted by DavidTC View Post
Well, my last idea didn't fly, but since then, I've discovered something very interesting: CC reads existing book in its directory when it connects to Calibre, and pulls the metadata from Calibre about those books. This almost lets me do what I want, in that I can point CC's 'Default folder' at a cloud-synced copy of the Calibre database directory, and then initiate a Calibre connection, and tada, it magically finds all the books and gives them records in CC. Eventually. After a really long time.
And it really doesn't work very well.

Auto-scan is an old feature that depends on the fact that books in CC's folder were put there by CC's wireless device. It was added because for 3 years or so we didn't have any way to backup and restore CC's database.

Why put there by CC? Because CC reads the metadata out of the books (epubs only). When a book is sent from calibre to CC via the WD connection, the metadata in the book is updated to what is in calibre at that moment. The most important bit is the book's UUID, which permits CC and calibre to agree later on which book in the calibre library matches which book in CC's library.

If the book "appears out of the sky" (as they do in the cloud connection) then the metadata in the book is almost certainly wrong. They only way it is up-to-date is if the user did something like "polish" in calibre on all the books so that the metadata is updated in the copies in calibre's library. If a book is not up-to-date then there is a very good chance that the book will not match the correct book in calibre's library. Or perhaps not match any book at all.

In addition, it works only for epubs. All other formats are uploaded to calibre for analysys.
Quote:
This shouldn't be too hard for a cloud connection. Please note I'm not asking to update all metadata, which you seem to have concerns about, just *adding* records when the book file is there without a record, exactly like the Calibre wireless device connection.
First, a bit of a peeve. Don't tell me what is hard and what isn't hard. I know my app (4 years of development) and calibre (6 years of development) inside out, and I am capable of estimating the complexity and work to do X.

What you are asking for is very similar to what we have already discussed and that I said I am considering. The cloud connection (no matter which one) has calibre's metadata.db in hand, so there is no reason at all to look in the books. Instead, CC would need to traverse the two databases (CC and calibre), find all books in calibre's db, then add them to CC, copy and all. This is a bit easier than updating metadata because one need not mess around with last-modified dates and the like.

One problem that has raised its head from time to time: the cloud connection cannot get the values of calibre custom columns "built from other columns". The values are not in calibre's metadata.db. People who use such columns will need to connect as a wireless device so that calibre and CC can fetch/update the values.
Quote:
Of course, the question is, would anyone but me use this? I think, yes. A good deal of people wish to keep all their books on their device already, and while it might not have occurred to them, the amount of sync tools on Android are pretty large, with all sorts of configuration. No more worries about updating.
This is the fundamental question. At the moment it appears that less than 5% of CC users use *any* cloud connection. One hard statistic I have: the number of devices connected to Dropbox is less than 2% of the devices actively running CC. My guess is that the number of people using the local cloud connection is less than 100, but I don't know that.

I need data to decide what to do. For this reason and many others, I have added gathering of some statistics to CC V5. With this information I will have a better idea of how many people use what and do what. Of course, people can opt out, skewing the stats. People who opt out in effect won't exist.
Quote:
Additionally, reading the wishlist, there appear to be a lot of people who want CC to read existing books in its directory, and the solution is always 'connect to Calibre'. I suspect at least a few of those people are using the old 'Set up a folder device inside dropbox, dump all the books in it' trick to get books out (Which is what a lot of people did pre-CC.), at which point they'd benefit to switching to what I described above.
There are three basic requests that CC doesn't handle, and won't handle. If you look through the reviews, several 1-star reviews make mention of these.
  • Divorce CC from calibre. Let me import my books from where ever, edit metadata, and the like. This is in effect duplicating calibre. I have no interest in going there.
  • Make CC a R/W client of calibre, with the ability to update calibre's metadata. This I would love to do, but the complexities are such that I wouldn't finish before entropy consumes the universe.
  • And one irrelevant to the discussion, add a reader app to CC. I see no reason at all to do this, given the number of apps to choose from.
My bottom line: I am not going to start some hacking adventure until I am convinced that there is a large enough audience. I have been surprised at how few people use the cloud connection. At the moment I am not at all convinced that even 0.1% of CC users have libraries on an SD card, but I willing to be wrong. With this level of take up, doing anything special with all the attendant risk doesn't make much sense.
chaley is offline   Reply With Quote