Calibre indicates book not on device, but it is

[webdoyenne]

Found this wonderful site while searching for an answer to my issue. Quickly registered, and here I am. Hi, you-all, from St. Petersburg, FL.

Still learning the ins and outs of calibre. Currently using it with iPad. Have generally had good luck converting files to epub and sending them to the iPad from calibre. Am seeing something odd today, however.

One of the books I transferred over is lacking the green checkmark to indicate that it's on my "device." The book is fer-shure on my iPad; I can open and read it. But calibre doesn't think it's there. Is there any way I can change this setting manually in calibre to let it know this particular book is alive and well on my iPad.

Not a big deal, generally speaking, since the book is there and I can access it. But would calibre not know that?

[chaley]

Originally Posted by webdoyenne

Found this wonderful site while searching for an answer to my issue. Quickly registered, and here I am. Hi, you-all, from St. Petersburg, FL.

Still learning the ins and outs of calibre. Currently using it with iPad. Have generally had good luck converting files to epub and sending them to the iPad from calibre. Am seeing something odd today, however.

One of the books I transferred over is lacking the green checkmark to indicate that it's on my "device." The book is fer-shure on my iPad; I can open and read it. But calibre doesn't think it's there. Is there any way I can change this setting manually in calibre to let it know this particular book is alive and well on my iPad.

Not a big deal, generally speaking, since the book is there and I can access it. But would calibre not know that?

Calibre matches the title and all the authors to determine if the book is the same. If there is any deviation between them, then they won't match. This can happen if you edit one of them in calibre to correct spelling, or if the iPad changes them for some reason, or if the order of authors changes. Capitalization shouldn't matter.

Compare the title and authors letter-by-letter. If they are identical, then work will be required to determine why they don't match. That will probably require the good offices of GRiker.

[GRiker]

I can think of one scenario where this might happen. Have you edited the title since adding the book to calibre?

G

[GRiker]

Originally Posted by webdoyenne

One of the books I transferred over is lacking the green checkmark to indicate that it's on my "device." The book is fer-shure on my iPad; I can open and read it. But calibre doesn't think it's there.

This happens with the iPad if you edit the title or the author of the book before sending it to the device.

The solution is to use Convert E-books to convert the book from EPUB>EPUB format before using Send to device to upload the book. The conversion writes the edited metadata to the epub.

G

[chaley]

Originally Posted by GRiker

This happens with the iPad if you edit the title or the author of the book before sending it to the device.

Unless the iPad does something different from the other devices, the metadata for a book is updated when it is copied. Calibre makes a temp copy of the book in the library, updates the metadata, and then hands that copy to the device driver.

It is possible that if the book is burdened with DRM, then no update will take place.

Of course, if a title or author is changed after the book is transferred, or if the book wasn't transferred by Calibre, then matching could easily fail.

[itimpi]

I thought the other devices wrote a calibre.metadata file to the device to help with matching the files? Since it would be normal for the process of writing to the device to update the metadata within the file that is being sent to the device, the file inside the Calibre library is quite likely to be different to the one on the device.

As it is working via iTunes, I am not sure if the iPad driver is working in that way? If not the details for deciding the 'on-device' status may be different, and it might be necessary to force the copy of the file inside the Calibre library to have its metadata updated before sending it to the iPad.

[chaley]

Originally Posted by itimpi

I thought the other devices wrote a calibre.metadata file to the device to help with matching the files?

Yes, it does, except for the ipad (but glancing at the code it appears to do something else). However, that doesn't change the situation. If you send a book to a device, then change the title, the metadata on the device will differ from the library. Devices that use .calibre.metadata use a UUID (unique ID) to match books that have been detected as the same in the past. I don't know if the iPad driver does this, or even can do this.

Since it would be normal for the process of writing to the device to update the metadata within the file that is being sent to the device, the file inside the Calibre library is quite likely to be different to the one on the device.

Yes. That is why checking if the book files are identical doesn't work.

As it is working via iTunes, I am not sure if the iPad driver is working in that way? If not the details for deciding the 'on-device' status may be different, and it might be necessary to force the copy of the file inside the Calibre library to have its metadata updated before sending it to the iPad.

Not sure what you are saying, beyond not being sure about how the iPad driver works (and I join you on that). What I am sure of is that calibre matches books against the library using the metadata from the calibre DB, not the book in calibre's library. The question is whether the iPad driver writes a book with updated metadata to the iPad when the book is sent. Of course, there is also the question about whether DRM will get in the way.

GRiker is certainly the expert on the iPad. Believe him before you believe me.

[GRiker]

Here's the iPad scoop.

I need to keep a copy of the uploaded file in both iTunes (on the host) and iBooks (on the iDevice). As a result, I can't use the temporary file with the updated metadata, because iTunes won't find it during a future calibre session when the temporary file is long gone. (This applies when the user has chosen to use their own storage, rather than having iTunes make its own copy of the transferred book). As a result, I have to use the real database copy, whose metadata is unaltered, causing the mismatch.

The initial implementation of the driver insisted that the user allow iTunes to make a copy, but folks who already had a substantial iTunes library using their own storage complained.

The challenge is all of the permutations - manual or automatic syncing, local or referenced storage.

I can add code that would rewrite the OPF file to rewrite the current metadata before uploading the file, but this seems wrong as it would be altering the original calibre copy. However, I could easily rationalize it by noting that iTunes is adding a plist to that very same file.

Input welcome.

G

[GRiker]

I've discussed with KG, I'm going to try doing a silent metadata update on the database copy before uploading. Makes me wince, but I don't see a better solution.

G

[chaley]

Originally Posted by GRiker

I've discussed with KG, I'm going to try doing a silent metadata update on the database copy before uploading. Makes me wince, but I don't see a better solution.

Out of curiosity, why doesn't UUID matching work? The UUID is in calibre's OPF file. When you connect to the iPad and get a list of books, are you getting partial metadata from somewhere?

[GRiker]

When I connect, I get two lists of books - the books in the iTunes library, and the books on the iDevice. The books may have come from the iBookstore, the user manually adding them in iTunes, or through the calibre interface. The metadata exposed through the automation interface is minimal, and doesn't include everything from the OPF file.

I've got the metadata-rewriting code working, just testing it now. It looks like it will work.

G

[itimpi]

Originally Posted by GRiker

I've discussed with KG, I'm going to try doing a silent metadata update on the database copy before uploading. Makes me wince, but I don't see a better solution.

Personally I would not see this as a big issue on an ePub file where it is easy to update the metadata without disturbing any of the formatting. There is a precedent set in the fact that the Viewer also silently alters the ePub file.

[kovidgoyal]

Originally Posted by itimpi

Personally I would not see this as a big issue on an ePub file where it is easy to update the metadata without disturbing any of the formatting. There is a precedent set in the fact that the Viewer also silently alters the ePub file.

It is annoying, as it breaks calibre's promise of not modifying your file (including metadata) unless you reconvert. But that's the price you pay if you decide to use Apple products.

[GRiker]

Originally Posted by chaley

Out of curiosity, why doesn't UUID matching work? The UUID is in calibre's OPF file. When you connect to the iPad and get a list of books, are you getting partial metadata from somewhere?

I revised the matching algorithm to use the UUID, thanks for suggesting this. I had it in there at one point, then changed it for some reason. I think it's all going to work as expected.

G