Kindle Device Library Duplication in Calibre for 'on device' listings.

[Mandabar]

So I have this issue on my kindle. I don't know why It does this, but it does play havoc with trying to keep track of what I have on the device and what I have in my library. It lists almost every book on the device as matching a single book on my library:

And this is my device list, after updating the cached () metadata to the device:
(Note 281 books)

If I delete the calibre metadata file on the kindle, I will at least have the listing on the device side be more accurate even if the library side won't change.

The more accurate device side listing:
But the library side doesn't change. Actually, the newest book added tonight is now missing from the 'correct' not duplicates section, now 1-4 (which makes sense, I deleted the metadata file): (Note, now 282 books)

I don't remember how this issue started, bout 3-4 months back. At this point I'm thinking the only solution might be to wipe everything and do it all again. Which would be... ugh.

No way to update the calibre metadata file to be based on the device side information or to link it to the in library stuff. Or to fix it?
====
On an unrelated note, let me throw out a few other questions I have that might save me from making another topic elsewhere. Not quite calibre related for one of them, so feel free to ignore.

1. I have three libraries, one for some more explicit content, one for the largest big thick fat books like cookbooks, robert jordan, or comic books and my normal main library. It used to be my main library was the main/first library I would see when I connected with my OPDS ebook reader thru the built in calibre content server. Not anymore. I have to selected it manually or it defaults to my large file library.

How can I change what is the default library for the content server without restricting my username ONLY to my main library?

2. Kindle Paperwhite Documents, Send to kindle. Why does it seem the syncing, or whispersync or whatever amazon calls it doesn't' seem to be working between my paperwhite signature edition and my kindle for android app? They don't sync progress, bookmarks or notes between them which is freaking weird. Yes sync is turned on for both devices.

Edit: Updated to use attachments instead of imgur links per Jon's guidance.

[JSWolf]

@Mandabar Please do not use imgr or any image host. One problem is the images will eventually expire and if the thread relies on the images, it becomes useless. The way to do it is to attach the images and then use the paperclip icon to insert the thumbnail. Thanks.

Imgur is the worst host as in the UK, it's banned and we have a number of UK users. Also, it's overloaded a lot. So even if you are allowed in, you can't see anything.

[JSWolf]

One thing to do is remove all of the eBooks on your Kindle. Then add them back in a little at a time. Each time making sure they are correct. It could be a case of you updated the metadata of all of your eBooks on your Kindle with the metadata from that one eBook.

But it's best to start over with the eBooks on your Kindle. I suggest you do 10 at a time to see that it's working properly.

[Mandabar]

Originally Posted by JSWolf

@Mandabar Please do not use imgr or any image host. One problem is the images will eventually expire and if the thread relies on the images, it becomes useless. The way to do it is to attach the images and then use the paperclip icon to insert the thumbnail. Thanks.

Imgur is the worst host as in the UK, it's banned and we have a number of UK users. Also, it's overloaded a lot. So even if you are allowed in, you can't see anything.

Huh. Interesting. I've never had an Imgur image deleted, unless I deleted it myself. Then again, they are uploaded to my account and not anon.
The being banned part is a whole different issue. yeah. I'll try to give em an upload.

Edit: The post has been updated to use attachments instead JSWolf.

Originally Posted by JSWolf

One thing to do is remove all of the eBooks on your Kindle. Then add them back in a little at a time. Each time making sure they are correct. It could be a case of you updated the metadata of all of your eBooks on your Kindle with the metadata from that one eBook.

But it's best to start over with the eBooks on your Kindle. I suggest you do 10 at a time to see that it's working properly.

I feared that would be the answer. I think at one point the metadata file on the kindle got deleted during my testing (And yes, it will deleted sideloaded books if you stay off wifi too long. Like Bezos is punishing you. I mostly stick to send to kindle now, giving amazon all that juicy data of just what books I be reading that I didn't buy from them...) and that is probably the root cause of all of this. I imagine Calibre only knows what is on the device, metadata linked wise, if it put it there and keeps records of it. Even though it technically is, and can, read the contents of the device it won't update itself based on that less than primary source of data. Or that's my rambling thoughts on it.

[JSWolf]

Don't use Send to Kindle. Let calibre convert to KF8 or KFX depending on what format you want to use. Calibre can convert to a KF8 or KFX on the fly so you don't end up with multiple formats of the same eBook. That's for ePub as the source.

[Sirtel]

Originally Posted by JSWolf

Don't use Send to Kindle. Let calibre convert to KF8 or KFX depending on what format you want to use. Calibre can convert to a KF8 or KFX on the fly so you don't end up with multiple formats of the same eBook. That's for ePub as the source.

And Amazon will delete those books, unless you keep wifi always off or on.

[jhowell]

Originally Posted by Mandabar

If I delete the calibre metadata file on the kindle, I will at least have the listing on the device side be more accurate even if the library side won't change.

Deleting that file causes the metadata to be re-extracted from the books on your Kindle. The result demonstrates that the metadata on the Kindle itself is correct and is being extracted correctly.

The problem appears to be related to calibre's matching of those books to your library. I suspect that the one book that is being matched to many on your device has been been corrupted within the calibre library in a way that causes it to incorrectly match.

I suggest deleting that book from your library. Then delete the metadata.calibre file from your Kindle and see if that corrects the problem.