View Single Post
Old 12-04-2016, 07:57 AM   #3
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 12,447
Karma: 8012886
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by Steve_Haley View Post
The logs describe a situation which doesn't make sense; they show you successfully sending some 4500 books in batches of about 700. Then, on your next connection, when CC and Calibre exchange information their respective libraries, Calibre states that it does not recognise any of the books in CC. This is baffling, as it's stating it doesn't recognise the books that it sent over itself. The connection process then fails shortly after this step.
The WD driver keeps a cache of the metadata of the books that it has sent to a device. This cache is per-device, and contains all the metadata for the books. When a device connects, CC sends a very small amount of information to calibre describing what is on the device. The WD driver checks the information against the cache, then asks CC on the device to send metadata for any books that are not in the cache.

If a book is not in the cache then calibre does what is called "book matching". It attempts to match books on the device with books in calibre's library using various metadata items such as the UUID (unique ID), the author and title, and the calibre ID. This process can fail to match a book, leaving books in calibre's device view that are not marked as "on device".

Bottom line: having the books not match what calibre's cache introduces two problems. The first: the requirement that CC resend all the metadata back to calibre is a process that takes a lot of time. The second: calibre fails to match the books to the library, making the on-device column wrong and defeating any metadata updating. Book matching can also take a long time (especially on 32-bit calibre), which might cause network timing problems.

I can think of 3 ways that cache problems can happen. All of them relate to how calibre's wireless device driver maintains which books on the device match which books in the library.
  1. The WD driver stores the cache in a hidden directory. On Windows this is C:\Users\<username>\AppData\Local\calibre-cache. Calibre could fail to write the cache file because of lack of space or incorrect permissions, or some "cleaner" could be removing the cache file(s).
  2. The cache file is tied to a specific device. That association is made the first time a device connects to calibre. If CC is reinstalled then the device association is forgotten and must be made again. There is no chance that CC and calibre will reuse the same ID, so the cache file will not be used.
  3. Calibre is running out of memory, causing the cache file to be corrupted. This is really only a possibility if a 32-bit version of calibre is being used.
If we are in case 2, where CC has been reinstalled thus invalidating the cache, the best approach is to uninstall CC, erase all the books on the device, reinstall CC, then resend the books. This is especially true of the iOS version of CC supports "scan on connect" where CC looks for books that are not in its database.

If we are in case 3 then the solution is to run 64-bit calibre on a 64-bit operating system.

Case 1 is more difficult. If the cache files are being erased then one must find what program is doing that erasing and stop it. If permissions are incorrect then something fundamental is broken.
chaley is offline   Reply With Quote