Merge existing formats into New records when adding new books

[BetterRed]

@camelx - I add books into a New Books library, I then 'do-what takes' to get the author names and titles 'correct'.

Then I use the Find Duplicates->Find Library Duplicates feature to check if New Books and my Main library have any duplicates. If there are any duplicates, I resolve by doing side-by-side eyeball comparisons - I run a calibre-server against my Main library to facilitate easy access.

Added : immediately after running Find Duplicates I select all non-duplicates and move them (Copy and delete) to the Main library. After which New Books will only have the duplicates.

When I was doing the initial build of my Main library, I would import ~50 'books' into New Books. Not for any technical reason, but that was a quantum that I could get done-and-dusted within a couple of hours - which is the limit of my attention span these days. I'm now adding around 20 'books' a day to my library, following the same process.

I sometimes want to keep both copies of a 'book'. That's easy enough if the 'new' version is in a different format. But, if they're the same format then I 'deal' with the duplicate as described here Calibre and Evernote

My 'books' are not commercial publications - they're PD documents, often the same text is published severally - revisions, errata, abridgment etc. FWIW I also handle the same 'book' in different languages via links to Evernote Notes in the Main library book entry.

BR

[cybmole]

Originally Posted by eschwartz

Usually, we are a friendly and helpful lot here.

well, responding to your "oh so friendly" attacks and put downs....

it was the implication , now withdrawn, that someone was updating batches of 2000 books, regularly, that I red flagged,

because there are no legit cases for someone obtaining such a large batch of improved commercial books, ( outside of academic papers/ fan fiction ). I know of no legit sites who sell books in bundles of that size.

but as we've now established that no one here is doing that, you'll have to go pick a fight somewhere's else.

if I call someone out for apparent piracy or kleptomania, it's because the posting points that way e.g. penniless student needs help with organizing half a million books / avid collector can't live with the fact that their e-reader can only store 10,000 books.... .

as for relative politeness, I'll let the totality of your posts v mine on all threads over the last 24 ours speak for that.

anyway have a nice weekend. I'll be spending mine not arguing with you.

[camelx]

Good enough, and again understood your stand after I re-read.

It's a shame this has gone this direction when I just wanted to see if it was possible to get added functionality.

Oh well, I don't want to fight and I can see your point. This has not been a passive-aggressive intentional stand on my side. I initially snapped back because it was not what I was about. I agree to just leave it.

[JSWolf]

I have to agree to cybmole. It does seem rather fishy updating 2000 eBooks in one go with them all being legit. The odds are against them all being legit.

[kovidgoyal]

Sorry, I'm rather busy right now, so I dont have time to read this thread, but from a quick once over of the first post, doesn't the existing option

Ignore duplicate incoming formats

do what you want? If not, why not?

[camelx]

I have said that I was using as an example and was a bad choice on my part. I give up, I'm out..............S#$T!

[camelx]

My understanding anyway...

Ignore- says to ignore duplicate formats and merge new formats into the "existing" records. What I was looking for, (just the opposite), is merge existing formats "not" found in the new record into the new records, and delete the old record.

[kovidgoyal]

You want the new duplicate formats to create a new record with the metadata from the old record and you want to delete the old record?

If that is the case, then overwrite existing will do what you want. It wont create a new record, but the effect is the same. You end up witha single record for the book, with updated formats.

[Dr. Drib]

Moderator Notice

Ok - That's it: If any member is going to stay out of the thread, THEN DO SO. Don't tease us.

Any further negative comments, attitudes, sly interrogatory comments, bad language, assumptions - from anyone - will be deleted.

AND TO BE VERY CLEAR ABOUT THIS: PIRACY WILL GET YOU BANNED.

[camelx]

When I have been referring to new records, I was talking about the ebooks that were being added, and previously existing records are the ones that are in the database prior to adding the new records. Both become duplicates when added, sorry I wasn't so clear. So back to my quick scenario of adding 4 new records that already exist in my database.

1) When I add the four new books, I want to keep all their metadata and book formats - yes they will be new records
2) they will be duplicates with the previously existing four records records that were already in the database
3) All I want to have done now is merge all book formats from the previous existing that do not exist in the new records.

Example:

In my database I have the Space Odyssey series, four books -
2001, 2010, 2061 and 3001 - they all have existing metadata for this example I'll call "old metadata", with book formats of epub and mobi

I'm adding four new Space Odyssey books I purchased online -
2001, 2010, 2061 and 3001 - I want them to come in as NEW duplicate records keeping their "new Metadata" and the keeping book format of epub
(EVERYTHING stays with the new records that are added. The only things that is done for merging is bringing the mobi format into the newly added records because this did not exist in the newly added records. The new records has all their new meatadata and their epubs and now the merged mobi format from the previously existing records. Then delete the older previously existing records)

They only work at this point for Calibre is:
1) identify that these are duplicate records
2) compare book formats of the newly added records with the previously existing records and
3) ONLY merge Book formats from the previously existing records that did NOT exist in the newly added records into the newly added records.
4) Then delete the previously existing records

I end up with no duplicates.
My four Space Odyssey books 2001, 2010, 2061 and 3001 have all the metadata of the newly added records, the new epub from those newly added records, and finally the merged in mobi format from the older existing records. The older existing records are deleted after pulling out the mobi format.

[itimpi]

Originally Posted by camelx

When I have been referring to new records, I was talking about the ebooks that were being added, and previously existing records are the ones that are in the database prior to adding the new records. Both become duplicates when added, sorry I wasn't so clear. So back to my quick scenario of adding 4 new records that already exist in my database.

1) When I add the four new books, I want to keep all their metadata and book formats - yes they will be new records
2) they will be duplicates with the previously existing four records records that were already in the database
3) All I want to have done now is merge all book formats from the previous existing that do not exist in the new records.

Example:

In my database I have the Space Odyssey series, four books -
2001, 2010, 2061 and 3001 - they all have existing metadata for this example I'll call "old metadata", with book formats of epub and mobi

I'm adding four new Space Odyssey books I purchased online -
2001, 2010, 2061 and 3001 - I want them to come in as NEW duplicate records keeping their "new Metadata" and the keeping book format of epub
(EVERYTHING stays with the new records that are added. The only things that is done for merging is bringing the mobi format into the newly added records because this did not exist in the newly added records. The new records has all their new meatadata and their epubs and now the merged mobi format from the previously existing records. Then delete the older previously existing records)

They only work at this point for Calibre is:
1) identify that these are duplicate records
2) compare book formats of the newly added records with the previously existing records and
3) ONLY merge Book formats from the previously existing records that did NOT exist in the newly added records into the newly added records.
4) Then delete the previously existing records

I end up with no duplicates.
My four Space Odyssey books 2001, 2010, 2061 and 3001 have all the metadata of the newly added records, the new epub from those newly added records, and finally the merged in mobi format from the older existing records. The older existing records are deleted after pulling out the mobi format.

As Kovid said, if you set the handling of duplicate formats to be overwrite do you not get the effect you want as any existing formats on the existing record are replaced and everything else is left intact? The process will be:

- Identify potential duplicates
- Accept them to replace any existing formats with the new ones leaving other formats and metadata intact.

No need to create new records or to delete the old one.

[camelx]

Thanks for your reply!

Unless I'm misunderstanding you, I don't want to leave the metadata intact from the original records in the database. I want the metadata of the new added records coming in.

[kovidgoyal]

So do it like this:

1) Create a new empty library
2) Add your books to it
3) Now add the books from the old library, using add recursively, one book per sub folder with the duplicates setting set to ignore.

Voila! You now have a merged library with both metadata and formats from the new files

[camelx]

Wow Kovid, it works perfectly, thank you very much for that!!!

(would still be a nice addition one day, to bring them straight into the main library)

But this way works and one could argue that not only that, but it is much safer with the intermediate steps.

Thanks again.