Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Library Management

Notices

Reply
 
Thread Tools Search this Thread
Old 09-05-2015, 02:10 PM   #1
digital_carver
Junior Member
digital_carver began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jun 2015
Device: Kindle PW
Download options for Identifier (isbn, asin, etc.) metadata

It seems like every metadata provider has their own identifier number (asin, google, goodreads, etc.), and they send that in along with the other pieces of metadata. Currently, Calibre doesn't allow not downloading this piece of metadata (apparently for some internal technical reason?), which means that in many cases the existing metadata is overwritten by some custom piece of identifier.

For example, I'm right now in the middle of going through my bulk metadata download results, and in 90% of the cases that require change, the change is that the local "mobi-asin" is about to be overwritten by "google" or "goodreads" identifiers. I wish to use the tags and such metadata from these providers, so disabling them entirely is not a solution. So in every case I tab to the existing identifier field, Ctrl-c, shift-tab twice to get back to downloaded identifier, and paste it there (I wish to merge, not overwrite, so the arrow button doesn't help.)

If it's not possible to disable the identifier metadata download, could an option be added to merge the existing identifier with the new ones, if the new ones don't contain an instance of the same identifier? (ie., merge existing local "mobi-asin:yyy" if downloaded data is "goodreads:xxx", but overwrite local "mobi-asin:yyy" if downloaded data is "goodreads:xxx,mobi-asin:zzz").

Is something like this already possible through some preference or plugin? If not, please consider adding it as an option.
digital_carver is offline   Reply With Quote
Old 09-05-2015, 03:16 PM   #2
theducks
Well trained by Cats
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 29,688
Karma: 54369090
Join Date: Aug 2009
Location: The Central Coast of California
Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A
Which is why I usually use Ctrl-D instead of the one built into Metadata editor. You see before (on right) and proposed (on Left) with an arrow to overwrite proposed with the old (Except you cant force it blank).
Blank proposed, always gets the old

Remember to configure EACH metadata source
theducks is offline   Reply With Quote
Advert
Old 09-05-2015, 04:20 PM   #3
digital_carver
Junior Member
digital_carver began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jun 2015
Device: Kindle PW
> You see before (on right) and proposed (on Left) with an arrow to overwrite proposed with the old

Yep, that's the screen I'm in. The issue is that I don't want to overwrite the identifier in most cases, rather wish to merge them: for eg., when the download has asin and google identifiers, my local file has isbn and mobi-asin already, the best of both worlds would be to merge them. But currently the only way to do that is to manualy copy and paste and add commas and all that, and do this again and again for every book.

I guess the original idea was that identifier would be a single piece of data, but now because of several sources we have, it's actually a combination of multiple pieces of metadata. So it doesn't always make sense to "update" or "correct" the local existing identifier with the downloaded one. That's the reasoning behind my suggestion above: treat the identifiers individually rather than as a single piece of string during metadata download, so that for eg. local "asin" would be overwritten by remote "asin", but local "isbn" wouldn't be affected by remote "asin" download, instead just getting merged with it.
digital_carver is offline   Reply With Quote
Old 09-05-2015, 04:50 PM   #4
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,457
Karma: 26645808
Join Date: Mar 2012
Location: Sydney Australia
Device: none
@digital_carver - FWIW - the Ctrl/D form of Metadata download (which can be used on a single book, as well as bulk) has a long press option on the IDs column [] button to Merge the values.

BR
BetterRed is online now   Reply With Quote
Old 09-05-2015, 08:03 PM   #5
digital_carver
Junior Member
digital_carver began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jun 2015
Device: Kindle PW
@BetterRed While looking at why my Calibre didn't have that long-press affordance at the place you mention, I found that I was running an ancient version of Calibre - 1.48, which seems to be the last .deb-packaged version of Calibre on any Linux repositories! I had assumed my regular 'apt-get upgrade's would have taken care of keeping Calibre updated, never knew that method had been abandoned. I'm now downloading the latest version through the recommended Linux install method.

And what you mentioned certainly a nice-to-have feature, but probably takes the same amount of time as manually copy-pasting it using keyboard shortcuts takes for me. But I'll give it a whirl once I get the new Calibre set-up with my preferences. (Don't want to just copy the config directory between major version changes, since that's already unsupported and might mess things up.)
digital_carver is offline   Reply With Quote
Advert
Old 09-05-2015, 08:32 PM   #6
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,457
Karma: 26645808
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by digital_carver View Post
@BetterRed While looking at why my Calibre didn't have that long-press affordance at the place you mention, I found that I was running an ancient version of Calibre - 1.48, which seems to be the last .deb-packaged version of Calibre on any Linux repositories! I had assumed my regular 'apt-get upgrade's would have taken care of keeping Calibre updated, never knew that method had been abandoned. I'm now downloading the latest version through the recommended Linux install method.

And what you mentioned certainly a nice-to-have feature, but probably takes the same amount of time as manually copy-pasting it using keyboard shortcuts takes for me. But I'll give it a whirl once I get the new Calibre set-up with my preferences. (Don't want to just copy the config directory between major version changes, since that's already unsupported and might mess things up.)
I would try using the existing config data, if you run into problems you can always reset it by running the Welcome Wizard - it's at the bottom right of Preferences.

I'm likewise not fond of mouse clicking - long or short. You can focus on the Identifiers button with Alt+I, Tab, Tab but then what - I don't think there's a keyboard equivalent of long press, in Windows at least.

BR

Last edited by BetterRed; 09-05-2015 at 08:34 PM.
BetterRed is online now   Reply With Quote
Old 09-05-2015, 09:42 PM   #7
digital_carver
Junior Member
digital_carver began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jun 2015
Device: Kindle PW
Quote:
Originally Posted by BetterRed View Post
I'm likewise not fond of mouse clicking - long or short. You can focus on the Identifiers button with Alt+I, Tab, Tab
Yup, that's basically what I did: Alt-I, tab, tab, Ctrl-C, shift-Tab, shift-Tab, End, comma, Ctrl-V, Alt-N

Went through my library of ~700 books this way, sometimes doing this, sometimes Alt-R, Alt-R, Enter (to reject), sometimes just Alt-I, delete (when my local data was a superset of the downloaded stuff). With some instrumental music running in the background, it was practically a Zen state.

Having to do this manually also helped me reject some pretty significant mistakes in the downloaded metadata - data of wrong book with somewhat similar title, sometimes even wrong author - so I guess even with such a merging feature I can't just let it do things automatically. (Though I wonder if my older version was to blame - I did these a while before discovering I was a few years out of date, so perhaps there's better algorithms in current versions.)
digital_carver is offline   Reply With Quote
Old 09-05-2015, 10:21 PM   #8
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Funny, I always thought the identifiers would be merged by themselves. It does seem the logical way to do things.

And doing a metadata download right now, and scrutinizing the value of the metadata before and after, shows me I am right -- the mobi-asin is preserved.


mobi-asin will never be downloaded -- it is calibre-specific, used to store the value of the exth 113 MOBI header (a/k/a ASIN). So if you have ever downloaded metadata for a book, and that identifier is still there, it must've been preserved... and I have many such books.


I cannot promise what outdated and downstream-patched versions of calibre will do (yes, calibre is patched downstream by many distros, which sometimes breaks functionality. e.g. cherrypy and content servers with paswords.)
eschwartz is offline   Reply With Quote
Old 09-05-2015, 11:22 PM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 43,771
Karma: 22666666
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Long click the arrow button and you will get a popup menu with an option to merge the identifiers.
kovidgoyal is online now   Reply With Quote
Old 09-06-2015, 08:14 AM   #10
digital_carver
Junior Member
digital_carver began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Jun 2015
Device: Kindle PW
Quote:
Originally Posted by eschwartz View Post
Funny, I always thought the identifiers would be merged by themselves. It does seem the logical way to do things.

And doing a metadata download right now, and scrutinizing the value of the metadata before and after, shows me I am right -- the mobi-asin is preserved.
I too tried it out now, and it does seem to already do what I suggested: overwrite similar identifiers, merge dissimilar ones. Neat. (I don't know if this was true of the ancient version I was using during the previous download, but it works in my current 2.37.1 Calibre.)

In fact, merging seems to happen with Tags too, which (afaik) seems undocumented. The existing tags remain, and the new tags get added additionally. For eg., with "The Autobiography of Benjamin Franklin", this was the tags metadata before download:

Quote:
Franklin, Statesmen -- United States -- Biography, Politics
This was the tags shown in the left side during "Review downloaded metadata":

Quote:
History, Biography, Classics, Politics, Non-Fiction
And this is the final tags list after pressing Ok on that dialog:

Quote:
Classics, non-fiction, Biography, Politics, Franklin, Statesmen -- United States -- Biography, History
As you can see, it's a merge rather than an overwrite.

The Configure Metadata Download dialog has only an "Append comments" option related to this, nothing about tags or identifiers, so it's not really obvious what all is going on in what field during the update.
digital_carver is offline   Reply With Quote
Old 09-06-2015, 01:06 PM   #11
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
Identifiers and tags in calibre manifest as lists.
Comments are a single string of HTML.


Rule of -- lists get merged.
eschwartz is offline   Reply With Quote
Old 09-06-2015, 07:18 PM   #12
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,457
Karma: 26645808
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by eschwartz View Post
Identifiers and tags in calibre manifest as lists.
Comments are a single string of HTML.

Rule of -- lists get merged.
- state of mind

Bulk download Tags and Identifier columns:

When I do a left mouse button click on the blue ← button (or hit space if it has focus) the existing values replace the downloaded values, which is what the button tooltip implies it will do - Discard the downloaded value for : <column name>

The same text appears as the first item in the dropdown list I get when I do a long left mouse button press on blue ← button.

To get values to merge I must do a long left mouse button press on blue ← button and select the second item in the dropdown list - Merge <column name>

Aside : I'd rather have two buttons on the list columns - blue ← button (Discard downloaded values) and aqua jaws button (Merge values). IMO it would be less confusing and consistent with the other blue ← buttons; they would also be keyboard-able . . .

BR
BetterRed is online now   Reply With Quote
Old 09-06-2015, 07:28 PM   #13
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
@BR,

I am aware there is a right-click option to merge, but at least for me (and @ digital_carver it would seem), simply leaving it alone ALSO merges them.
eschwartz is offline   Reply With Quote
Old 09-06-2015, 08:00 PM   #14
BetterRed
null operator (he/him)
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 20,457
Karma: 26645808
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Quote:
Originally Posted by eschwartz View Post
@BR,

I am aware there is a right-click option to merge, but at least for me (and @ digital_carver it would seem), simply leaving it alone ALSO merges them.
I hadn't tried right button click, I read somewhere it needed a long left button mouse press, being mouse (and touch) averse that is not something I would think of doing 'naturally'. But I did try the windows menu key which always does what a right click does everywhere else in and out of calibre - except on these two buttons

Nevertheless for me at least a click on the Tags or Identifier blue ← does what the tooltip says it will do - discard the downloaded values - what do you see in the tooltip ?

It guess it could be a Windows 10 aberration - but I would be gobsmacked if it is.

BR
BetterRed is online now   Reply With Quote
Old 09-06-2015, 08:35 PM   #15
eschwartz
Ex-Helpdesk Junkie
eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.eschwartz ought to be getting tired of karma fortunes by now.
 
eschwartz's Avatar
 
Posts: 19,422
Karma: 85397180
Join Date: Nov 2012
Location: The Beaten Path, USA, Roundworld, This Side of Infinity
Device: Kindle Touch fw5.3.7 (Wifi only)
I don't click on it. I leave the right side and the left side intact -- I click Review, eyeball the results, and go straight on to Accept, do not pass Go, do not collect $200 for clicking things.

Clicking once on the arrow will reject the downloaded values and present the "result" for approval.
Clicking twice merges values and presents the results for approval.
Clicking zero times merges values after you click Accept.
eschwartz is offline   Reply With Quote
Reply

Tags
identifier, merge metadata, metadata, metadata download, preferences

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Metadata download: Only ISBN (example) AMF Library Management 6 04-29-2019 10:17 PM
Download metadata from ASIN CaitlinShaw Library Management 3 06-17-2015 12:56 AM
Metadata download by ISBN? vedri Library Management 3 07-31-2013 06:00 PM
Mobi-asin and metadata desertblues Calibre 6 01-20-2013 09:43 AM
Configure Metadata download for ISBN overridden by Preferences meme Library Management 11 09-21-2011 11:47 PM


All times are GMT -4. The time now is 04:13 AM.


MobileRead.com is a privately owned, operated and funded community.