Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Devices

Notices

Reply
 
Thread Tools Search this Thread
Old 08-01-2011, 10:14 AM   #1
misling
Zealot
misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.
 
Posts: 106
Karma: 510000
Join Date: Nov 2010
Device: Kindle
Kindle device problem with both library and collection

I'm having a weird problem with Calibre and my Kindle.

My Kindle has ~2000 books on it. About half are mobi and the other half are azw with a few azw1 (topaz). All the books on the device are in a collection.
I have the latest version of Calibre (I download the new version every Saturday). I have the Kindle collections plugin installed (I have others as well, but probably unrelated to this problem)

1. I create a new empty calibre library 'Kindle Device Stuff' and switch to it.
2. I connect my Kindle via USB
3. I wait until the device button appears and all the books on the Kindle are listed under it. The library is still empty at this point.
4. In the 'device' view screen use the "add to library" function to add all books from the device to the empty library.
5. I switch over to the library to see the books that have been added (they are all added). Here is where the FIRST problem shows:
PROB 1: All the azw/azw1 books are marked as being on the device in Main memory with a checkmark in the column. All the mobi books are NOT marked as being on the device (even though that is where they were loaded into the library from).
6. I run the collections plugin option to import the collections from the Kindle. Here is where the SECOND problem shows.
PROB 2: All the mobi books are shown as being in a Collection. All the azw/azw1 books are NOT shown as being in a collection.

End result from the library view:
all Mobi books are 'not on the device' but 'in a collection'
all azw/azw1 books are 'on the device' but 'not in a collection'

Real situation: all books of all types are 'on the device' AND 'in a collection'

My purpose is to move books around between collections. This situation makes that impossible.

Anybody got any ideas?
misling is offline   Reply With Quote
Old 08-05-2011, 09:40 AM   #2
misling
Zealot
misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.
 
Posts: 106
Karma: 510000
Join Date: Nov 2010
Device: Kindle
Anybody?
Not being able to tell what's on my Kindle is really messing things up.
I keep accidentally adding duplicate books to my Kindle from Calibre since half of the books on the device don't show as being on the device.
And of course the collections plugin is useless too when it doesn't report half of the collections on the device.
misling is offline   Reply With Quote
Advert
Old 08-05-2011, 11:26 AM   #3
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
More info needed, please

Hi misling.
I'm no expert, but heard your cry for help.
First of all, I need more info.
Secondly I suggest we concentrate on getting Kindle and Calibre in sync since that's the basis for it all.

This is a new problem that suddenly occured, did it use to work (with other libraries, etc etc ) ?

Which Kindle do you have ?
Kindle FW version ?
OS version ?

Mobi books on Kindle: Are they DRM'ed ? (copy protected) ?
Your azw and azw1 DRM'ed ?

What's the size of you metadata.calibre on Kindle ?

Are ALL books on your Kindle transferred via Calibre ?!!

It might also be possible that your metadata.calibre on Kindle is corrupt, but hard to say. I had what might be a similar situation in february, and ended up biting the bullet (start from scratch) BUT YOU shouldn't do that yet !!.

I'll try to do some tests tonight/tomorrow to see if I can recreate your scenario, I hope.

Brds Per

Last edited by pchrist7; 08-05-2011 at 01:02 PM. Reason: Forgot ....
pchrist7 is offline   Reply With Quote
Old 08-06-2011, 06:05 AM   #4
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
Using plugboards ?

Hi again
Did some tests, as far as I dared to go

I use send to device plugboard to add series info to title when sending to my K3. When importing books into new, empty library, the books are imported, but title now includes series info if book is on a series.
If book is in a series (and included in title), then calibre does not show the book as being on device.
If i change the book title back to the original (remove series info), restart calibre, the book shows as being on device !

Does this help ?

Waiting for info from you, misling

Brgds Per
pchrist7 is offline   Reply With Quote
Old 08-06-2011, 06:33 AM   #5
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 9,864
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
Quote:
Originally Posted by pchrist7 View Post
I use send to device plugboard to add series info to title when sending to my K3. When importing books into new, empty library, the books are imported, but title now includes series info if book is on a series.
If book is in a series (and included in title), then calibre does not show the book as being on device.
I don't have a Kindle but I always thought the books placed on your device using plugboards would show up in calibre's library as on device. At least they always show as being on device on all Sony devices.
DoctorOhh is offline   Reply With Quote
Advert
Old 08-06-2011, 07:26 AM   #6
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
Quote:
Originally Posted by dwanthny View Post
I don't have a Kindle but I always thought the books placed on your device using plugboards would show up in calibre's library as on device. At least they always show as being on device on all Sony devices.
Hi
They do show up as on device in calibre.
This issue here is when importing/copying books from kindle to a new calibre library.
Some books show correct status, some don't.
Need more info from OP !

Bgrds Per
pchrist7 is offline   Reply With Quote
Old 08-06-2011, 08:35 AM   #7
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 11,703
Karma: 6658935
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by pchrist7 View Post
I use send to device plugboard to add series info to title when sending to my K3. When importing books into new, empty library, the books are imported, but title now includes series info if book is on a series.
If book is in a series (and included in title), then calibre does not show the book as being on device.
If i change the book title back to the original (remove series info), restart calibre, the book shows as being on device !
There is probably an interaction between the setting of "Read metadata from files on device" in the Kindle plugin, "read metadata from file contents rather than file name" in preferences -> adding books, and importing from the Kindle. If the one in Add Books is set, then calibre will read the metadata from the book and get the plugboard-modified data. If the one in the device plugin is not set, then the device system will read metadata from the path, which is not affected by the plugboard. Result: titles do not match, so ondevice is blank.

Book matching between a book on the device that calibre did not put there and calibre's library is approximate. It depends on very-close-to-exact matches between authors and title, something that is often not the case, especially with purchased DRMed books.

If calibre put the book on the device, and if no one has messed with the .metadata.calibre file or changed any part of the name of the book file on the device, then book matching is done using a unique ID. I have no idea whether putting a book into a collection on the Kindle changes some part of the file path. If it does, then book matching by UUID will fail and calibre drops back to title/author matching.
chaley is offline   Reply With Quote
Old 08-07-2011, 12:04 AM   #8
pchrist7
Addict
pchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animalspchrist7 is kind to children and small, furry animals
 
pchrist7's Avatar
 
Posts: 385
Karma: 6514
Join Date: Aug 2010
Location: Denmark
Device: Kindle 3 3G+Wifi, Oasis
That explains it

Quote:
Originally Posted by chaley View Post
There is probably an interaction between the setting of "Read metadata from files on device" in the Kindle plugin, "read metadata from file contents rather than file name" in preferences -> adding books, and importing from the Kindle. If the one in Add Books is set, then calibre will read the metadata from the book and get the plugboard-modified data. If the one in the device plugin is not set, then the device system will read metadata from the path, which is not affected by the plugboard. Result: titles do not match, so ondevice is blank.

Book matching between a book on the device that calibre did not put there and calibre's library is approximate. It depends on very-close-to-exact matches between authors and title, something that is often not the case, especially with purchased DRMed books.

If calibre put the book on the device, and if no one has messed with the .metadata.calibre file or changed any part of the name of the book file on the device, then book matching is done using a unique ID. I have no idea whether putting a book into a collection on the Kindle changes some part of the file path. If it does, then book matching by UUID will fail and calibre drops back to title/author matching.
Thanks chaley !
I think you explained it all.
Adding books to collections changes nothing besides collections.json file AFAIK.

Brgds Per
pchrist7 is offline   Reply With Quote
Old 08-07-2011, 07:28 AM   #9
meme
Sigil developer
meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.
 
Posts: 1,275
Karma: 1101600
Join Date: Jan 2011
Location: UK
Device: Kindle PW, K4 NT, K3, Kobo Touch
As pchrist7 says, adding books to collections only changes the collections.json file. It does not update the metadata in books or change their titles or pathnames, etc. It does not change what is stored in calibre.

Whether a book shows up as on-device in Calibre has nothing to do with the plugin. If some books are not showing up as on-device that are really on the device, then I suggest you look at resolving this first before trying to resolve any collections issue.

The plugin gets the information about what is on device directly from calibre these days, and doesn't use the metadata.calibre file.

I wonder if renaming metadata.calibre (on the Kindle) and then restarting calibre to force it to rebuild metadata.calibre will help. Of course with 2000 books this could take a very long time! And if the calibre options are set as chaley described it might not change anything anyway.
meme is offline   Reply With Quote
Old 08-07-2011, 09:09 AM   #10
misling
Zealot
misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.
 
Posts: 106
Karma: 510000
Join Date: Nov 2010
Device: Kindle
Sorry had to travel off on business. I'll try to answer the questions as far as I understand them.
Quote:
This is a new problem that suddenly occured, did it use to work (with other libraries, etc etc ) ?
I'm not sure if this just started happening. My original library is about 8000 books and until I created the new empty library I may not have noticed if all the books didn't show as being on the device. I'd just added the collections plugin and my purpose in creating the new library was to have a place to work with just the books on the device and their collections, so of course I noticed it then Before this point I was doing collections by hand.

Quote:
Which Kindle do you have ?
Kindle 3 3G

Quote:
Kindle FW version ?
3.1

Quote:
OS version ?
Windows Vista Ultimate

Quote:
Mobi books on Kindle: Are they DRM'ed ? (copy protected) ? Your azw and azw1 DRM'ed ?
Half and half. The azw/azw1 probably about 75% copy protected, the mobi stuff is not.

Quote:
What's the size of you metadata.calibre on Kindle ?
4,131 KB

Quote:
Are ALL books on your Kindle transferred via Calibre ?!!
no. The amazon azw/azw1 stuff downloads from Amazon. The mobi stuff is sideloaded via USB from the main Calibre library. I imported them all from the Kindle (USB) to the Calibre empty library in order to do collections on them.

Last edited by misling; 08-07-2011 at 09:27 AM.
misling is offline   Reply With Quote
Old 08-07-2011, 09:14 AM   #11
misling
Zealot
misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.
 
Posts: 106
Karma: 510000
Join Date: Nov 2010
Device: Kindle
OK so if I'm following this:
Quote:
There is probably an interaction between the setting of "Read metadata from files on device" in the Kindle plugin, "read metadata from file contents rather than file name" in preferences -> adding books, and importing from the Kindle.
then what I need to do is check where the preferences are set to see if they are all doing the same thing for add books and the Kindle plugin (and maybe the collection plugin?). I'll try that next.

Thanks for the help!

I'm tempted to just redo the kindle from scratch, as somebody said above.
misling is offline   Reply With Quote
Old 08-07-2011, 10:49 AM   #12
meme
Sigil developer
meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.meme ought to be getting tired of karma fortunes by now.
 
Posts: 1,275
Karma: 1101600
Join Date: Jan 2011
Location: UK
Device: Kindle PW, K4 NT, K3, Kobo Touch
Renaming the metadata.calibre file on the Kindle doesn't redo your kindle from scratch - it just removes the cache file that Calibre uses. By renaming it, you force calibre to read through all the books on your device to get their information so it can rebuild the cache file to save time each time your device is connected. But if the settings below aren't in sync I don't think it will help.

Check Preferences -> Adding Books - is Reading metadata from file checked or unchecked

Check Preferences -> Plugins, then expand Device Interface Plugins, then click Kindle 2/3 Device Interface, then click Customize Plugin. Is Read metadata checked or unchecked?
meme is offline   Reply With Quote
Old 08-08-2011, 11:00 AM   #13
misling
Zealot
misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.
 
Posts: 106
Karma: 510000
Join Date: Nov 2010
Device: Kindle
I'll check those settings when I get home if I can find time to boot the laptop - I don't spend much time on the computer at home, and right now I'm at work.
Thanks for noting the path to the device settings under the plugin, don't know if I would have found that without instructions.

Oh, and I wasn't talking about removing the metadata or whatever. I was talking about doing a device restore to factory default, and reloading my books. Wouldn't fix this problem, I don't think, but I've been considering it because the books on the device are all f'd up due to duplicates being loaded on.
misling is offline   Reply With Quote
Old 08-09-2011, 12:04 PM   #14
misling
Zealot
misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.misling ought to be getting tired of karma fortunes by now.
 
Posts: 106
Karma: 510000
Join Date: Nov 2010
Device: Kindle
OK. The 2 preferences are set the same and apparently that is not the problem.
I took screenshots but I don't see how to put them in here. It just asks for a url. My pictures aren't on the web, they are on my desktop. I don't have time to find a picture upload service and sign up for it, have to get back to work.

Here is the whole sequence.
Create an empty library.

Check Preferences -> Adding Books - is Reading metadata from file checked or unchecked? CHECKED

Check Preferences -> Plugins, then expand Device Interface Plugins, then click Kindle 2/3 Device Interface, then click Customize Plugin. Is Read metadata checked or unchecked? CHECKED

Plug in the Kindle. Wait for the Device screen to show up and fill.
Go to Device screen.
Highlight all.
Rightclick and 'Add to Library'.

In the library page, and also on the device page, Half the books, the azw/azw1 show "Main" under "on device" and half, the mobi, do not.

I hadn't noticed it the last time, but this time in the library page a few of the mobi were recognised - about 10 of them.

Also, on the library page the azw/azw1 are in a collection and the mobi are not. On the device page, the mobi are in a collection and the azw/azw1 are not. This is just weird.

All the books are on the Kindle device - that's where I built the library FROM. Both are really in a collection on the Kindle.
misling is offline   Reply With Quote
Old 08-09-2011, 10:12 PM   #15
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 9,864
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Nexus 7
Quote:
Originally Posted by misling View Post
I took screenshots but I don't see how to put them in here. It just asks for a url. My pictures aren't on the web, they are on my desktop. I don't have time to find a picture upload service and sign up for it, have to get back to work.
Upload service not required. Edit your post, go to the Advanced editor, click the paperclip to attach a file and upload your screenshots as attachments.
DoctorOhh is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Free (Kindle/Nook) Investor's Library (Collection) arcadata Deals and Resources (No Self-Promotion or Affiliate Links) 4 08-15-2011 03:21 AM
calibre removes the collection on my device that I made for my library books LadyHeather Calibre 5 01-06-2011 10:01 AM
Library Name as a Collection on Device averyml Calibre 3 11-30-2010 02:20 PM
Kindle DX US problem: Device Password Unavailable steve.chan.bj Amazon Kindle 2 07-26-2010 12:22 PM
Collection Problem JeffElkins Calibre 6 11-03-2008 06:33 AM


All times are GMT -4. The time now is 07:17 AM.


MobileRead.com is a privately owned, operated and funded community.