Register Guidelines E-Books Today's Posts Search

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 08-25-2013, 11:39 PM   #1
icallaci
Guru
icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.
 
Posts: 817
Karma: 6528026
Join Date: Sep 2012
Device: Kobo Elipsa
Calibre 1.0 - Duplicate formats

I have my Calibre prefs set to overwrite an existing format if I add the same book to my library. I noticed tonight that I have a bunch of duplicate formats. It looks like they have all appeared since I upgraded to version 1.0. The book titles, authors, etc. are all exactly the same, because I copy the .epub file to a flash drive, fix the typos and then re-add the book to Calibre by clicking on the Edit Metadata button and then clicking on the Add Books icon. I ran Library Maintenance tonight, and the duplicates showed up, but since I have my prefs set to overwrite, should this be happening? It has not ever happened before, so...
icallaci is offline   Reply With Quote
Old 08-25-2013, 11:41 PM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 45,364
Karma: 27230406
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
To be clear, you're saying that if you do the following steps:

0) copy out the .epub file from a book somewhere else
1) Click edit metadata for the book
2) Click the add book button and choose the file copied out in step 0
3) Click OK

You end up with a duplicate epub file? What exactly is duplicated? You have two epub files in the book record, with different names? Or something else?
kovidgoyal is online now   Reply With Quote
Advert
Old 08-25-2013, 11:49 PM   #3
icallaci
Guru
icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.
 
Posts: 817
Karma: 6528026
Join Date: Sep 2012
Device: Kobo Elipsa
Quote:
Originally Posted by kovidgoyal View Post
To be clear, you're saying that if you do the following steps:

0) copy out the .epub file from a book somewhere else
1) Click edit metadata for the book
2) Click the add book button and choose the file copied out in step 0
3) Click OK

You end up with a duplicate epub file? What exactly is duplicated? You have two epub files in the book record, with different names? Or something else?
Yes, I end up with two epubs in the folder (not sure about any difference in the naming, sorry--I ran Library Maintenance and deleted the dups). I just tried it again without making any changes to the file and did not get a duplicate, so it appears that either the file size must be different or something else must be different in order to generate a duplicate. I'm sorry I don't have a lot of info for you at this point. I will keep an eye on it, and do some more testing to see if I can narrow down what is happening. Thanks.

Edit: Not sure if it matters, but the OS is Windows 7, 64-bit.

Last edited by icallaci; 08-26-2013 at 12:33 AM. Reason: Additional info
icallaci is offline   Reply With Quote
Old 08-26-2013, 12:00 PM   #4
icallaci
Guru
icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.
 
Posts: 817
Karma: 6528026
Join Date: Sep 2012
Device: Kobo Elipsa
Quote:
Originally Posted by kovidgoyal View Post
To be clear, you're saying that if you do the following steps:

0) copy out the .epub file from a book somewhere else
1) Click edit metadata for the book
2) Click the add book button and choose the file copied out in step 0
3) Click OK

You end up with a duplicate epub file? What exactly is duplicated? You have two epub files in the book record, with different names? Or something else?
Sorry, it was late last night, and I misread your post. I do not show two epub files in the book record. I show only one epub in Calibre's book record, but I have two epub files in the file system folder. Calibre's book record points to the correct file, but I am left with an additional file that is orphaned in the file system. I will try to provide additional information later today, after I do some testing.
icallaci is offline   Reply With Quote
Old 08-26-2013, 02:04 PM   #5
icallaci
Guru
icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.
 
Posts: 817
Karma: 6528026
Join Date: Sep 2012
Device: Kobo Elipsa
Well, of course I cannot make this happen again (after seeing approximately 30 duplicates over the course of several days). The only thing I can think of that might have some bearing on the problem, is that the duplicate files had their names truncated from 36 characters to 31 characters, as noted in this thread: https://www.mobileread.com/forums/sho...d.php?t=220936, which seems to be new behavior. I will continue to watch for this problem, and if I can make it happen again, I will write the steps down exactly to see if anyone else can replicate this problem.
icallaci is offline   Reply With Quote
Advert
Old 08-26-2013, 03:42 PM   #6
tilia
Evangelist
tilia ought to be getting tired of karma fortunes by now.tilia ought to be getting tired of karma fortunes by now.tilia ought to be getting tired of karma fortunes by now.tilia ought to be getting tired of karma fortunes by now.tilia ought to be getting tired of karma fortunes by now.tilia ought to be getting tired of karma fortunes by now.tilia ought to be getting tired of karma fortunes by now.tilia ought to be getting tired of karma fortunes by now.tilia ought to be getting tired of karma fortunes by now.tilia ought to be getting tired of karma fortunes by now.tilia ought to be getting tired of karma fortunes by now.
 
tilia's Avatar
 
Posts: 432
Karma: 1720909
Join Date: Mar 2011
Device: Voyage, K3
I have noticed something similar.

When I add books to my kindle, I also make a new version of the catalog I keep on the device. With the new database, the new version of the catalog shows up as a new entry in the book list in calibre.

Both entries in the list points to the same files in the library-folder. And removing the catalogue from the device, removes the Ondevice-tick from both entries.

The duplicate entry disappears with a restart of calibre.
tilia is offline   Reply With Quote
Old 08-26-2013, 10:36 PM   #7
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 45,364
Karma: 27230406
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
@tilia: That is indeed a bug and will be fixed in the next release, it does not however have anything to do with icallaci's problem. https://github.com/kovidgoyal/calibr...e7fa93e6c8921c

Last edited by kovidgoyal; 08-27-2013 at 12:09 AM.
kovidgoyal is online now   Reply With Quote
Old 08-27-2013, 02:10 AM   #8
icallaci
Guru
icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.
 
Posts: 817
Karma: 6528026
Join Date: Sep 2012
Device: Kobo Elipsa
OK, I think I know what is going on. The duplicate format problem is definitely related to the length of the book title and/or filename. I just added new versions of several books and only got duplicates of the ones with long file names. Books with short titles don't show the problem. Here are steps to replicate the problem:

1. Pick an epub book with a long title (not sure how long it has to be, so I suggest 36+ characters). The epub must have been added to Calibre before Calibre 1.0.

2. Copy the .epub file at the OS level to a flash drive and make whatever changes you want to it. (I use Sigil, but I have no idea if this contributes to the problem or not. I doubt it.) I also am not sure if file size has anything to do with the problem, so I suggest making enough changes to affect the size of the file.

3. Open Calibre (upgraded to 1.0), select the epub, and click on the Edit Metadata button. (Make sure your preferences for adding books are set to "overwrite existing duplicate formats.")

4. Add the new version of the edited epub by clicking on the red rectangle with the white plus sign, finding the file on the flash drive, and double-clicking it.

5. Go to the directory where Calibre stores books and find the book you just added. You should (I hope) find two epubs with similar names. Only one of these (the correct one, thank goodness) shows up in Calibre. The old version is still in the directory, orphaned.

I hope this helps. (Using Windows 7, 64-bit.)
icallaci is offline   Reply With Quote
Old 08-27-2013, 02:39 AM   #9
DoctorOhh
US Navy, Retired
DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.DoctorOhh ought to be getting tired of karma fortunes by now.
 
DoctorOhh's Avatar
 
Posts: 9,897
Karma: 13806776
Join Date: Feb 2009
Location: North Carolina
Device: Icarus Illumina XL HD, Kindle PaperWhite SE 11th Gen
Quote:
Originally Posted by icallaci View Post
5. Go to the directory where Calibre stores books and find the book you just added. You should (I hope) find two epubs with similar names. Only one of these (the correct one, thank goodness) shows up in Calibre. The old version is still in the directory, orphaned.
It looks like you have solved the mystery. Everything is working as planned except you would expect the old file to be deleted. I suspect previously this wasn't an issue since it would have over-written the old file with the exact same name.

If you run Check library under Library Maintenance does it give you the option to delete these orphaned books?
DoctorOhh is offline   Reply With Quote
Old 08-27-2013, 02:44 AM   #10
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 45,364
Karma: 27230406
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
@icallaci: Ah, yes that makes sense. The problem is caused by a performance optimization, where when replacing a format the original is not deleted first, but just directly replaced. This causes the orphan problem when the filename for the new format is different from the filename for the old format, as in the case of the changed path length calculation. Fix will be in the next release.

You can just ignore the orphans or remove then via Library Maintenance.
kovidgoyal is online now   Reply With Quote
Old 08-27-2013, 04:54 AM   #11
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 45,364
Karma: 27230406
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
And here is the fix: https://github.com/kovidgoyal/calibr...3d31b51196c4fb
kovidgoyal is online now   Reply With Quote
Old 08-27-2013, 09:42 AM   #12
icallaci
Guru
icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.
 
Posts: 817
Karma: 6528026
Join Date: Sep 2012
Device: Kobo Elipsa
Quote:
Originally Posted by kovidgoyal View Post
Thank you so much!
icallaci is offline   Reply With Quote
Old 08-27-2013, 09:44 AM   #13
icallaci
Guru
icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.icallaci ought to be getting tired of karma fortunes by now.
 
Posts: 817
Karma: 6528026
Join Date: Sep 2012
Device: Kobo Elipsa
Quote:
Originally Posted by DoctorOhh View Post
If you run Check library under Library Maintenance does it give you the option to delete these orphaned books?
Yes, Library Maintenance finds the orphaned books and lets me delete them. I do wish there was a "Select All" for the procedure, though.
icallaci is offline   Reply With Quote
Reply


Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Calibre formats and duplicate book entries jgt1942 Calibre 4 03-20-2013 09:55 AM
Finding and Deleting Duplicate Files of different formats dpayment General Discussions 19 10-19-2011 03:02 PM
Merging duplicate entries with different formats and slightly different metadata? maxsys Library Management 2 02-20-2011 04:19 AM
Auto-Merge Question re: Duplicate Formats barium Library Management 7 02-02-2011 11:31 PM
Duplicate books - multiple formats mranlett Calibre 5 09-26-2009 07:02 AM


All times are GMT -4. The time now is 10:42 PM.


MobileRead.com is a privately owned, operated and funded community.