Merge existing formats into New records when adding new books

[camelx]

This may be a request for new functionality, unless someone chimes in to show me how I can do this.

What is needed -

The ability when adding new books to keep the new book metadata and all it's book formats, but when duplicate books are found in the existing library, then ONLY pull book formats that do not exist in the new records.

Why would we need this -

there are times when I want to add new books where I know the metadata is better and maybe the book formats are cleaner than I had, but I don't want to lose a format I had in the existing records that is not in the new record.

Another example is where I buy a new series of books that have the series perfectly setup, I have duplicates of those books, but the series is all messed up. The new books I'm adding are only epub and my exisitng books with messed up series is mobi. I want all the metadata from the new books I'm adding to be what I end up with and merge the mobi from my duplicates.

What I have to do now to accomplish this -

simple example:
I want to add the perfectly structured Authur C. Clarke's Space Oddessy - 4 books.
I have existing 4 books in my library but series is not there, the covers are older and a lot of metadata is missing are not right, titles namescontaining series, etc. The existing format is mobi and some have epub, but they are many years old and those epubs are not all clean.

the new books I'm adding are perfectly formatted, series are named right and in order, covers are up to date and all metadata is much cleaner and format is epub, the epubs are formatted much better.

1) I add the new books (without automerger, to preserve the older records to get their epub formats manually). I now have four duplicates.

2) I right click on a new added book, series 1, and select similar books by author - only Authur C. Clarke books showing to keep it easy to work with.
3) I sort by series (my four new books are together)
4) I highlight the first book in series, then click the title column heading to sort by title. (this shows my first book in series right next to my existing book, 2001 in this case - new clean record and old record)
*****
5) THIS STEP is my main goal in my manual process - MY new record, series 1 book is highlighted and holding the control key, I highlight the existing old duplicate book). This now keeps all the metadata of new book and epub of new record, and adds only the mobi format from my old record and deletes that old record.
******

Please don't say I can highlight the old records and easily edit in bulk the series, or use the series manager, etc. I know that. I also know I could bulk update with download covers and metadata.

The above example was a very simple example. A common case is where I will be adding a couple thousand books with many series updates, better metadata, cleaner formatted books, etc.

Bottom line of goal -

I want to use everything the new book record has - metadata and book formats and ONLY add missing book formats of DUPLICATE records to the new record and delete the old record.

My proposal for new Calibre enhancement -

Preferences/adding books
checking the: automerge books if they already exist in the Calibre Library
add a new drop down: "Keep the new record metadata and formats and add only missing formats from duplicates".

-------------
Thanks for bearing with me on this. If I'm totally missing a way to do this, I'm sorry and please inform me. I really doubt I'm alone on this. I think this would be a powerful feature as a lot of us have library full of old records that could be cleaned up with such a feature. I hope Kovid will consider this!!!

[eschwartz]

If I understand correctly -- you want to be able to use a pre-existing standalone feature (select which book is the master copy during Merge) in the Add books ==> Auto-merge feature?

Personally, I would stay away from auto-merge, and handle it all manually.
I believe the automerge is intended only on the theory that you have a lovingly fine-tuned library, but got new formats. And the metadata inside a book is likely to be inferior to the cleaned-up metadata in your library.

[cybmole]

I love hearing about these folks who commonly buy 2000 books at a time, to add to their library, just s t to replace 2000 older ones. Torrent much, do you.
. I could do that if a) I had Money to burn , b) I was likely to live long enough to read them
now don't pretend they are all PD series, please. 2000 retail books, call it 10k dollars, is not something most folks buy commonly and frequently, just to have cleaner formatting ?

[cybmole]

Ps or would you care to retract on
A common case is.... a couple thousand books....

[PandathePanda]

Originally Posted by cybmole

I love hearing about these folks who commonly buy 2000 books at a time, to add to their library, just s t to replace 2000 older ones. Torrent much, do you.
. I could do that if a) I had Money to burn , b) I was likely to live long enough to read them
now don't pretend they are all PD series, please. 2000 retail books, call it 10k dollars, is not something most folks buy commonly and frequently, just to have cleaner formatting ?

It's not always piracy. My mom amassed 4800 books in 6 months from kindle. I recently had to go help a pensioner in town, she complained her "ebook" reader is too slow to open. Went and had a look, she's using K4PC, it took me 15-20 minutes of waiting time for it too open. Her collection is staggering. She tells me she spends between an hour to 4 hours "shopping" a day at amazon (using one of the daily free amazon book sites) her collection last time I looked was over 26K of free amazon books.

[eschwartz]

cybmole, your suspicions are unwarranted, unneeded, and offensive. Please do not post if you have nothing useful to contribute.

[cybmole]

Sorry, I was under the mistaken impression that this board was not in favour of ebook piracy.
I will happily retract, given an example of how one can commonly acquire batches of 2000 new improved e-books, that is all I am querying. Well that and how long it takes to read them.

[eschwartz]

And if you had a good reason to believe that the OP is pirating books, then the board might do something about it. Unfortunately for you and your crusade (past and present), you have no proof, and the usual rule is "innocent until proven guilty".

Just because it isn't the most common thing in the world, does not mean it does not happen. There are a number of legal ways to do so, and it is a given that there are at least a handful of people who do just that. In fact, if I remember correctly, this is pretty much a repeat of a previous conversation, and nobody was impressed with your logic then either. The board administrators certainly didn't feel the need to do anything about it.

You know what -- if you are so sure the OP is a pirate, put your money where your mouth is, and report the OP for piracy. Let us see whether the board admins agree with you.

[camelx]

1) thanks for comment on being care about automerge, I totally agree with that, but there are some cases where I know my existing library is worse.
2) as for 20k book comment, I did not mean that I would ever add anything in mass like that, I should have worded it if someone ever would, that this would be invaluable. You shouldn't make assumptions you have no idea about, you sound like a righteous jerk, thinking no one is going to slip anything past you. Do I have a some books that people shared with me and didn't pay for? yes I do, but not many, I don't want to cheat. Are you telling me you don't have even just one yourself? give me a break.
3) As for the comment about large libraries - I have a friend that spend incredible money on ebooks, he's obsessed with them. And no I don't have access to them or wouldn't even ask.

I still think it would be a nice option, just thought I'd ask.

[JSWolf]

I would be interesting in knowing how someone can get 2000 improved eBooks in one go to replace 2000 older versions without there being piracy.

[eschwartz]

Originally Posted by camelx

1) thanks for comment on being care about automerge, I totally agree with that, but there are some cases where I know my existing library is worse.

I am sure that can happen as well. To continue the recent theme there are people who legitimately bought large numbers of ebooks back in the early days of the market, when the quality wasn't always so good. Later they revamped their library by rebuying newer editions.

If memory serves correct, some of those people are extremely respected members of this board.


Anyway... I am not sure Kovid Goyal would want to add that as an automerge feature, especially given the Find Duplicates plugin, which can give you a lot of fine-grained control, allowing you to do binary or metadata compares, compare across physical libraries, step through the results one match at a time...

I would try that out.

2) as for 20k book comment, I did not mean that I would ever add anything in mass like that, I should have worded it if someone ever would, that this would be invaluable. You shouldn't make assumptions you have no idea about, you sound like a righteous jerk, thinking no one is going to slip anything past you. Do I have a some books that people shared with me and didn't pay for? yes I do, but not many, I don't want to cheat. Are you telling me you don't have even just one yourself? give me a break.
3) As for the comment about large libraries - I have a friend that spend incredible money on ebooks, he's obsessed with them. And no I don't have access to them or wouldn't even ask.

We do try to avoid talking about "shared" ebooks. My personal policy is to plead the fifth on general principle.

As I said, there are valid reasons to have a lot of books (even if you don't), including the last time I was able to track down guess-who making an also-false accusation: https://www.mobileread.com/forums/sho...cy#post2609853

Sorry you had to deal with this.

[cybmole]

1. I don't recall launching any past crusade. 2. The OP has clarified his meaning. .
3. Next time someone asks for help with metadata for half a million new books I will stay out of it .
Goodnight.

[camelx]

I sincerely apologize to all, as I corrected, I do have access to any large amount of books and I do not pirate books. I fully understand all who try to respect and protect this. I merely meant it as an example that is someone did have a huge amount of books they knew were clean, wouldn't it be helpful.

I'm not doing it and I threw the number out cause it sounded huge. I sure didn't want to start a bunch of crap around here and again, I do understand the concerns, my bad!

[eschwartz]

There is no reason to feel bad, you did nothing wrong.
Just like anywhere, MobileRead has a certain percentage of people who are overly critical of others.

Again, as an enthusiastic MobileRead member I apologize for the rude greeting you received. Usually, we are a friendly and helpful lot here.




Anyway, back on topic. Have you given the Find Duplicates plugin a try? It is more or less designed for your exact situation -- although it does not work automatically.

[camelx]

Yes I did, it's wonderful and safe and I'll continue to use it in these cases. Funny thing about Calibre is the more you use it, the more you are obsessed with getting all the data perfect. I've found good solutions to most of my work except the case I originally posted about. Thank you for your support and I should have been more careful in my use of examples.

I can sure now see where this would appear to be a pirating need, but it still would be valuable.