Merging records and ISBN

[Starson17]

The Merge function does not merge ISBN numbers. This was intentional, but it's recently been reported as a bug. Despite my (slight) desire to respond "Feature, not bug!" I'm inclined to change it to add ISBN into the merge function.

When I wrote Merge, I was in the middle of hundreds of merges and general metadata cleanup. When a record has an ISBN number, it overrides the author/title during single or bulk metadata fetch operations. You only get one record back during a fetch.

As a practical matter, here's what I was seeing:

1) If I was not doing a metadata fetch, I had no use for the ISBN. It didn't tell me anything.

2) If I was doing a metadata fetch, it was because the other metadata was bad and the ISBN prevented me from fetching multiple results and choosing the best. I was only able to fetch the single result corresponding to the ISBN.

I was constantly having to delete the ISBN (usually obtained as a result of a previous bulk metadata fetch) in order to get several results and find the best match of metadata.

When merging, I often had multiple single format records (created before the autosorting option was available) and one or two of them would have a bad ISBN from a previous bulk metadata fetch where the author or title was wrong. As a result, I preferred not to merge the ISBN.

All of this boils down to the fact that my ISBN numbers never came directly from the book. I always used the option to get metadata only from the filename, and the filename never had the ISBN in it.

That's the "Why" for this bug/feature. However, while I'm glad it did not merge ISBNs for me ... looking back on it, I suspect that this "feature" for me is more likely to be a "bug" for others. I'm inclined to fix it.

Before I do, I'm interested in any comments anyone wants to make. (It's not the sort of thing I think is likely to be worth an option checkbox.)

Any comments?

Thanks!

[theducks]

Originally Posted by Starson17

The Merge function does not merge ISBN numbers. This was intentional, but it's recently been reported as a bug. Despite my (slight) desire to respond "Feature, not bug!" I'm inclined to change it to add ISBN into the merge function.

When I wrote Merge, I was in the middle of hundreds of merges and general metadata cleanup. When a record has an ISBN number, it overrides the author/title during single or bulk metadata fetch operations. You only get one record back during a fetch.

As a practical matter, here's what I was seeing:

1) If I was not doing a metadata fetch, I had no use for the ISBN. It didn't tell me anything.

2) If I was doing a metadata fetch, it was because the other metadata was bad and the ISBN prevented me from fetching multiple results and choosing the best. I was only able to fetch the single result corresponding to the ISBN.

I was constantly having to delete the ISBN (usually obtained as a result of a previous bulk metadata fetch) in order to get several results and find the best match of metadata.

When merging, I often had multiple single format records (created before the autosorting option was available) and one or two of them would have a bad ISBN from a previous bulk metadata fetch where the author or title was wrong. As a result, I preferred not to merge the ISBN.

All of this boils down to the fact that my ISBN numbers never came directly from the book. I always used the option to get metadata only from the filename, and the filename never had the ISBN in it.

That's the "Why" for this bug/feature. However, while I'm glad it did not merge ISBNs for me ... looking back on it, I suspect that this "feature" for me is more likely to be a "bug" for others. I'm inclined to fix it.

Before I do, I'm interested in any comments anyone wants to make. (It's not the sort of thing I think is likely to be worth an option checkbox.)

Any comments?

Thanks!

Worms!

Huge can, of

I agree with your original assumptions and reasoning.
It is a PITA if you have the wrong ISBN (at least we now have a ISBN is syntactically Valid, indicator )

ISBN is needed to fetch covers, so if you have a cover, do you need the ISBN for other reasons?
The same title may have multiple ISBN's (because of ISBN rules), so the existing ISBN's may not match (and are NOT incorrect).

As I said.
Worms!

[Starson17]

Originally Posted by theducks

Worms!

Well, that's why I asked.

At the least, I should put a note in the Merge warning screen that tells the user about this "feature" if it isn't fixed as a "bug."

One option for the user is always to "select first" the record with the ISBN number prior to doing a merge. An ISBN in the first record is never changed.

This feature/bug only matters if the first selected record has no ISBN and one of the others has an ISBN. Otherwise the result is the same, no matter which way the code is written.

[chaley]

The first thing I do when dealing with metadata is remove the ISBNs. They never correspond to the physical book I have (I buy physical books all over the world). They just get in the way.

I have considered building a tweak that would prevent the box from ever being filled in, and/or adding a 'clear ISBN' check box to bulk metadata edit. Haven't done either because Kovid has said he will eventually build a general solution to the download metadata choice problem.

I suppose I am voting that you leave it like it is: not merge ISBNs. If you feel you need to change it, make the behavior tweakable.

[Starson17]

Originally Posted by theducks

It is a PITA if you have the wrong ISBN

One problem with the ISBN is that it doesn't appear in the columns of the GUI, so you can't easily see if you have an ISBN or not. Normally I don't care if it's there or not, but when I'm preparing to do a bulk metadata fetch, I look at the author and title to make sure they are correct. If there's a bad ISBN hidden in there, however, it will override the "correct" author/title.

[speakingtohe]

I prefer not to merge isbn
I would like to merge custom columns with the proviso that if first one selected is blank it gets overwritten, but maybe not
I use merge a lot and overall it is perfectly fine as it is.
Helen

[Starson17]

Originally Posted by speakingtohe

I prefer not to merge isbn ...
I would like to merge custom columns with the proviso that if first one selected is blank it gets overwritten, but maybe not

Well, that's 4 for 4 to leave it as is. I'm glad I posted. I personally like it as is, but it did seem a bit odd/inconsistent to leave out that one bit of metadata during a Merge. At this point I'm inclined to just update the warning to tell the user that ISBN is not merged.

I left a message in the bug report directing the bug reporter here, so perhaps he'll make some comments.

As to custom metadata, that was added to the Merge function a month or so ago. It's treated exactly like the type of non-custom metadata that the custom field is based on.

If a field in the destination record (first selected) is empty, it gets filled in. If it's not empty, it is never overwritten, but it will get additional metadata appended, where possible (without duplication). Tags and comment-like fields are two fields where additional data is appended, but checked to prevent duplication.

[Starson17]

Originally Posted by speakingtohe

I use merge a lot

By the way, the next release will have "Alt-M" as a keyboard shortcut for safe-merge (merge-keep others) as a companion to the "M" keyboard shortcut for a normal merge.

[theducks]

Originally Posted by Starson17

By the way, the next release will have "Alt-M" as a keyboard shortcut for safe-merge (merge-keep others) as a companion to the "M" keyboard shortcut for a normal merge.

Is it easy to do : Have a side by side conflict resolution GUI box appear with replace arrow buttons between the conflicted fields:

source => destination
(fancy tweak: tristate, per field, preference: Always, Never, Ask default: ask )

[Starson17]

Originally Posted by theducks

Is it easy to do : Have a side by side conflict resolution GUI box appear

When you Fetch metadata off the web, you're doing something similar to a Merge. You have metadata on the web and it may or may not overwrite the metadata in the record. I put together a simple "do/don't overwrite author/title" control, but a more comprehensive system would let you individually control whether ISBN is overwritten during fetch, whether title, but not author is overwritten, etc.

As Charles has mentioned, Kovid has this type of system on his todo list. Once he has done the hard work, it might be relatively straight forward to adapt it for controlling Merge in addition to Fetch.

[chaley]

A late posting, but it is probably worth mentioning that the custom field beta of calibre lets you make a column that displays ISBN (but read-only), and the search/replace in bulk edit works on them.

I have now removed all isbns in one swoop.

[speakingtohe]

When you Fetch metadata off the web, you're doing something similar to a Merge. You have metadata on the web and it may or may not overwrite the metadata in the record. I put together a simple "do/don't overwrite author/title" control, but a more comprehensive system would let you individually control whether ISBN is overwritten during fetch, whether title, but not author is overwritten, etc.

As Charles has mentioned, Kovid has this type of system on his todo list. Once he has done the hard work, it might be relatively straight forward to adapt it for controlling Merge in addition to Fetch.

I would dearly love to see a bulk overwriting fetch for individual items such as series.
When I started using calibre, series fetch wasn't implemented. I made up my own names or copied them from Fantastic Fiction. I have corrected many but still have quite a mishmash I am sure.
I live in hope
Helen

[theducks]

Originally Posted by speakingtohe

I made up my own names or copied them from Fantastic Fiction. I have corrected many but still have quite a mishmash I am sure.
I live in hope
Helen

Your "Made up names" have a chance of being more consistent .

Even the Publisher can't seem to do that.
Look at Baen's "Series" Web page.
Now look at the "Books by Author" front matter in the books.

What is the real Series Name?

Then there is the case, where you buy a book, not knowing it is going to be, Book 1 in a Series.
Or the Author that claims "Darkover is not a series", there is no correct reading order

I do believ, until Kovid and crew get the crystal ball plug-in working, that we will be stuc with re-visiting our entries and making adjustments

[speakingtohe]

I have found a few inconsistencies but mostly pretty. Downoading them sure beats going down an unnumbered list and counting for me anyway.

do believ, until Kovid and crew get the crystal ball plug-in working, that we will be stuc with re-visiting our entries and making adjustments

Don't tell me the crystal ball plug-in is broken. I wanted to borrow it for halloween.
Helen