[Android] Calibre showing 2 files on device for recently sent despite having 1 only

Dopedangel · 11-22-2016, 01:29 AM

I am having a problem with the latest version of calibre companion and calibre. Calibre shows that I have 2 books on device despite having only 1. I deleted those files resent but after disconnecting and connecting calibre companion it shows as having 2 files again on my device. Its only happening with the latest files sent. This started maybe yesterday. And the same error on 2 different devices.
Calibre companion version is 5.2.2.2 (351) and calibre version is the latest.

chaley · 11-22-2016, 02:59 AM

As I said in the other thread, I don't think this is related to the beta.

The first thing to do is to check that CC's database is consistent with the book files. Run CC's "Books with missing files" from the main menu. Fix any problems.

Next, connect to calibre. Assuming that the "problem" book is still there (one that shows 2 when only one is in CC), in calibre's library view delete the book. You will see a dialog asking if you want to delete from the device, from the library, or both. Pick device. You will now see a dialog showing the two books on the device. The 6th column contains the paths of the books on the device. These must be different. Note these paths. Now, on the device, use a file manager to verify that these paths point at actual books.

If there are actual books there then the book really does exist twice in CC. If one of the paths doesn't exist then we begin to focus on the problem. In this case, please do the following:

Connect CC to calibre.
Delete a problematic book from the device using calibre's delete function, not CC's delete function.
Disconnect CC.
Restart calibre in debug mode.
Reconnect CC.
Verify that the book is not on-device. If it is then stop here, sending me the calibre and CC debug logs.
If the book is not on-device then send that one book to CC. The book should now show on-device, 1 copy. If it does not, stop here and send me the CC and calibre debug logs.
Disconnect and reconnect CC.
If the book now shows more than one copy on the device, send me the CC and calibre debug logs.

Dopedangel · 11-22-2016, 04:11 AM

Okay I will try all this though same thing is happening in 2 different devices 1. My phone 2. My reader. If it had happened on one device I would could understand database getting corrupted.
Btw I was already deleting using calibre. And remove books on device shows 2 entries exactly the same on calibre

Sent from my SM-G925I using Tapatalk

chaley · 11-22-2016, 04:19 AM

Quote:

Originally Posted by Dopedangel

Okay I will try all this though same thing is happening in 2 different devices 1. My phone 2. My reader. If it had happened on one device I would could understand database getting corrupted.
Btw I was already deleting using calibre. And remove books on device shows 2 entries exactly the same on calibre

You are saying that the two books shown in calibre's delete dialog have *exactly* the same path? No differences in letter case? If so then the problem might be a corrupted cache file in calibre.

If you are running windows then the cache is in \Users\<your name>\AppData\Local\calibre-cache. Make sure you are not connected, then delete all the "wireless_device..." files. When you reconnect CC will send all metadata back to calibre, reinitializing the cache.

Dopedangel · 11-22-2016, 04:23 AM

Quote:

Originally Posted by chaley

You are saying that the two books shown in calibre's delete dialog have *exactly* the same path? No differences in letter case? If so then the problem might be a corrupted cache file in calibre.

If you are running windows then the cache is in \Users\<your name>\AppData\Local\calibre-cache. Make sure you are not connected, then delete all the "wireless_device..." files. When you reconnect CC will send all metadata back to calibre, reinitializing the cache.

sorry I edited my previous post

Dopedangel · 11-22-2016, 04:29 AM

I am using calibre portable deleted all the wireless cache files still the same problem. I deleted and re added a few other books and now those are also showing as having 2 files on my device.

chaley · 11-22-2016, 04:34 AM

Quote:

Originally Posted by Dopedangel

Okay I will try all this though same thing is happening in 2 different devices 1. My phone 2. My reader. If it had happened on one device I would could understand database getting corrupted.
Btw I was already deleting using calibre. And remove books on device shows 2 entries exactly the same on calibre

Sent from my SM-G925I using Tapatalk

The paths are different. One starts with the tag "!Science fiction" and the other with the tag "!Science Fiction". That shouldn't happen. I need debug logs from both CC and calibre to try to work out which program is making the mistake.

chaley · 11-22-2016, 04:38 AM

Question: is the option Settings / File names / Replace existing book files checked?

Dopedangel · 11-22-2016, 04:53 AM

Quote:

Originally Posted by chaley

Question: is the option Settings / File names / Replace existing book files checked?

The option is disabled meaning I can see the option but can't change it. Another thing the actual tag is Science fiction but the folder in my device that calibre companion makes is !Science Fiction. I have been using it this way for a long time I dont know what happened apart from installing the latest version of calibre and calibre companion. Have sent the debug file from my phone.

chaley · 11-22-2016, 05:20 AM

OK, I think I know what is going on.

The root cause is a tag, Science Fiction, had its case changed. I think that 'Fiction' became 'fiction'. This means that the folder on Android no longer has the right name.
Folder names in Android can be case sensitive, so sometimes the case change doesn't matter but sometimes it does.
You are not using CC templates, so none of CC's processing to detect case changing is being applied.
Your device is apparently using a case-insensitive file system. This means that when you send a book starting with "!Science fiction", Android will happily use the folder "!Science Fiction" without changing its case.
You have the setting Settings / Connecting to calibre / Wireless device connection / Scan on connect checked. This tells CC to check for books on the device that are not in the database. Because the case of the database file name is different from the case of the actual file name, the file name comparison is failing and CC is reporting new books with the different name.

The first thing to try is to turn off Scan on connect. I think that will fix it. If you are happy leaving it off then we are done. If, however, you want to use scanning then you will need to switch to CC templates. We can discuss this later if we need to.

Another possibility is to correct the case of the "Science fiction" tag back to "Science Fiction".

Dopedangel · 11-22-2016, 05:52 AM

I will change the tag to Science Fiction though I don't get why this started happening suddenly when before it was working okay. And it's the same thing on 2 devices with different versions of Android.
The only thing different is I updated calibre companion. I don't have access to my pc now will check back later if this solves the problem or not.
Thanks for the help.q

Sent from my SM-G925I using Tapatalk

Dopedangel · 11-23-2016, 01:06 AM

Changing the Tag fixed my problem thanks

chaley · 11-23-2016, 04:15 AM

Quote:

Originally Posted by Dopedangel

Changing the Tag fixed my problem thanks

Thanks for letting me know. Now all I need to do is find a way to prevent it from happening again. There isn't an obvious solution ...

chaley · 11-23-2016, 03:21 PM

I have changed CC to make the scan-on-connect ignore files that differ from books in CC's database only in case. This will fix the problem you are having and similar problems, such as when an author or title changes only in letter case.

I am not totally convinced that I am not creating new problems. For example, on some Android devices the file path "author-Title" really is different from "Author-Title". If on such a device a user puts a book differing only in letter case into the library folder then connects as a wireless device, with this change CC will ignore the book. My guess is that case changes in tags, titles, and authors are far more common than wanting a new book to appear from a scan with a letter-case difference. We will see ...

BTW: my guess is that you got into this situation because metadata download fetched tags differing in letter case from what was in your DB. If you accept the download then calibre will change the case for all books with that tag.

Dopedangel · 11-23-2016, 07:32 PM

Quote:

Originally Posted by chaley

I have changed CC to make the scan-on-connect ignore files that differ from books in CC's database only in case. This will fix the problem you are having and similar problems, such as when an author or title changes only in letter case.

I am not totally convinced that I am not creating new problems. For example, on some Android devices the file path "author-Title" really is different from "Author-Title". If on such a device a user puts a book differing only in letter case into the library folder then connects as a wireless device, with this change CC will ignore the book. My guess is that case changes in tags, titles, and authors are far more common than wanting a new book to appear from a scan with a letter-case difference. We will see ...

BTW: my guess is that you got into this situation because metadata download fetched tags differing in letter case from what was in your DB. If you accept the download then calibre will change the case for all books with that tag.

I don't fetch tag metadata as to me it's mostly useless. But I get metadata off books so maybe one of the recent books had a lowercase Science fiction which changed the tag for my whole collection. I have another tag that was Science Fiction.Mystery. I usually use title case for tags so am not sure how this one wasn't.
Thanks for the change but if it breaks something for other I don't mind if you go back to previous implementation.

Sent from my SM-G925I using Tapatalk

11-22-2016, 01:29 AM	#1
Dopedangel Wizard Posts: 1,791 Karma: 30548723 Join Date: Dec 2006 Location: Singapore Device: Boyue	[Android] Calibre showing 2 files on device for recently sent despite having 1 only I am having a problem with the latest version of calibre companion and calibre. Calibre shows that I have 2 books on device despite having only 1. I deleted those files resent but after disconnecting and connecting calibre companion it shows as having 2 files again on my device. Its only happening with the latest files sent. This started maybe yesterday. And the same error on 2 different devices. Calibre companion version is 5.2.2.2 (351) and calibre version is the latest.

11-22-2016, 02:59 AM	#2
chaley Grand Sorcerer Posts: 12,336 Karma: 8012652 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	As I said in the other thread, I don't think this is related to the beta. The first thing to do is to check that CC's database is consistent with the book files. Run CC's "Books with missing files" from the main menu. Fix any problems. Next, connect to calibre. Assuming that the "problem" book is still there (one that shows 2 when only one is in CC), in calibre's library view delete the book. You will see a dialog asking if you want to delete from the device, from the library, or both. Pick device. You will now see a dialog showing the two books on the device. The 6th column contains the paths of the books on the device. These must be different. Note these paths. Now, on the device, use a file manager to verify that these paths point at actual books. If there are actual books there then the book really does exist twice in CC. If one of the paths doesn't exist then we begin to focus on the problem. In this case, please do the following: Connect CC to calibre. Delete a problematic book from the device using calibre's delete function, not CC's delete function. Disconnect CC. Restart calibre in debug mode. Reconnect CC. Verify that the book is not on-device. If it is then stop here, sending me the calibre and CC debug logs. If the book is not on-device then send that one book to CC. The book should now show on-device, 1 copy. If it does not, stop here and send me the CC and calibre debug logs. Disconnect and reconnect CC. If the book now shows more than one copy on the device, send me the CC and calibre debug logs.

11-22-2016, 04:11 AM	#3
Dopedangel Wizard Posts: 1,791 Karma: 30548723 Join Date: Dec 2006 Location: Singapore Device: Boyue	Okay I will try all this though same thing is happening in 2 different devices 1. My phone 2. My reader. If it had happened on one device I would could understand database getting corrupted. Btw I was already deleting using calibre. And remove books on device shows 2 entries exactly the same on calibre Sent from my SM-G925I using Tapatalk Attached Thumbnails Last edited by Dopedangel; 11-22-2016 at 04:22 AM. Reason: added the snapshot of what I mean

11-22-2016, 05:20 AM	#10
chaley Grand Sorcerer Posts: 12,336 Karma: 8012652 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	OK, I think I know what is going on. The root cause is a tag, Science Fiction, had its case changed. I think that 'Fiction' became 'fiction'. This means that the folder on Android no longer has the right name. Folder names in Android can be case sensitive, so sometimes the case change doesn't matter but sometimes it does. You are not using CC templates, so none of CC's processing to detect case changing is being applied. Your device is apparently using a case-insensitive file system. This means that when you send a book starting with "!Science fiction", Android will happily use the folder "!Science Fiction" without changing its case. You have the setting Settings / Connecting to calibre / Wireless device connection / Scan on connect checked. This tells CC to check for books on the device that are not in the database. Because the case of the database file name is different from the case of the actual file name, the file name comparison is failing and CC is reporting new books with the different name. The first thing to try is to turn off Scan on connect. I think that will fix it. If you are happy leaving it off then we are done. If, however, you want to use scanning then you will need to switch to CC templates. We can discuss this later if we need to. Another possibility is to correct the case of the "Science fiction" tag back to "Science Fiction".

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Kindle Fire - book not showing on device within Calibre.	KevinCoed	Devices	11	08-08-2020 09:29 PM
Recently opened book not showing on home page	sparklemotion	Amazon Kindle	10	01-11-2014 11:14 PM
Calibre -> Android/Aldiko, covers for epub not showing in device	Backdraft	Devices	8	06-14-2011 11:33 PM
Calibre not showing Save to Device Icon	sissiromy	Calibre	0	12-20-2010 12:45 PM
Device Not Showing in Calibre	suipedali	Calibre	0	04-04-2009 02:57 PM

11-22-2016, 04:29 AM	#6
Dopedangel Wizard Posts: 1,791 Karma: 30548723 Join Date: Dec 2006 Location: Singapore Device: Boyue	I am using calibre portable deleted all the wireless cache files still the same problem. I deleted and re added a few other books and now those are also showing as having 2 files on my device.

11-22-2016, 04:38 AM	#8
chaley Grand Sorcerer Posts: 12,336 Karma: 8012652 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	Question: is the option Settings / File names / Replace existing book files checked?

11-22-2016, 05:52 AM	#11
Dopedangel Wizard Posts: 1,791 Karma: 30548723 Join Date: Dec 2006 Location: Singapore Device: Boyue	I will change the tag to Science Fiction though I don't get why this started happening suddenly when before it was working okay. And it's the same thing on 2 devices with different versions of Android. The only thing different is I updated calibre companion. I don't have access to my pc now will check back later if this solves the problem or not. Thanks for the help.q Sent from my SM-G925I using Tapatalk

11-23-2016, 01:06 AM	#12
Dopedangel Wizard Posts: 1,791 Karma: 30548723 Join Date: Dec 2006 Location: Singapore Device: Boyue	Changing the Tag fixed my problem thanks

11-23-2016, 03:21 PM	#14
chaley Grand Sorcerer Posts: 12,336 Karma: 8012652 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	I have changed CC to make the scan-on-connect ignore files that differ from books in CC's database only in case. This will fix the problem you are having and similar problems, such as when an author or title changes only in letter case. I am not totally convinced that I am not creating new problems. For example, on some Android devices the file path "author-Title" really is different from "Author-Title". If on such a device a user puts a book differing only in letter case into the library folder then connects as a wireless device, with this change CC will ignore the book. My guess is that case changes in tags, titles, and authors are far more common than wanting a new book to appear from a scan with a letter-case difference. We will see ... BTW: my guess is that you got into this situation because metadata download fetched tags differing in letter case from what was in your DB. If you accept the download then calibre will change the case for all books with that tag.