Calibre Metadata not same as EPub Meta - Page 2

eurasiarc · 10-11-2013, 05:21 PM

I just used Calibre to edit some tags and reconverted the books to epub format. The tags for these books appear in the book information in the Calibre reader for PC.

Then I used Calibre Companion to sync the metadata and also send a new epub book that contains the tags I added.

I can see the new tags and metadata for the book listings in Calibre companion, but none of the edited metadata are tags appear in the book information in the Montano library, which is set to search for books in the same folder on the external sdcard that is used to send books via Calibre Companion. Strangely some of the epub books in the Montano Library, which are exactly the same books in the Calibre Companion library have tags I have never seen before. A few books have as many as twenty or thirty tags, but none of the tags for epub books that I created using calibre are listed in the tags list in Montano.

Any ideas about what could be causing this?

Even the new epub book with tags that appear in Calibre and Calbre companion appears without any tags in Montano reader.

chaley · 10-11-2013, 05:59 PM

My guess is that mantano is doing some kind of import when it first sees a book, and it's not reimporting the book when it changes. Is there a function in mantano to force a database refresh? If not, then you might be forced to delete and reimport the book.

BTW: it is exactly this sort of problem that pushed us to make CC into a full fledged book manager with searching and grouping. Trying to understand all the ways that the reader apps manage metadata in their libraries was a lost cause.

Adoby · 10-11-2013, 06:21 PM

As I stated earlier you have to delete and reimport books in Mantano to make updated metadata show up. There is no refresh metadata function in Mantano.

CC gets the information about the tags from calibre, not from inside the books. Mantano gets the information about the tags from inside the books, not from calibre.

Refresh metadata in Mantano would have been a great feature. Perhaps even that Mantano check the timestamp of the books, in the background and automatically refresh metadata for updated books, and delete removed books and add new ones. Or even that Mantano detects and use a database with information saved by calibre in the top of the book folder. And rules for how the books should be added to different collections would also be nice. For instance you could have regular regular expressions and logical operators that map to a collection:

sci(ence)*\s?fi(ction)* OR space\s?opera maps the book to the collection "Science Fiction", if any tag match.

eurasiarc · 10-12-2013, 01:58 PM

Thanks for the latest information and suggestions on editing metadata in Calibre and getting the new metadata to appear in readers such as Mantano. I have been busy making changes in the tags for many epub books in my library before sending them to my device and deleting and reimporting them into Mantano to see if the new tags appear. (By deleting, I mean deleting the database entries for Mantano reader, not the books themselves.) When I get this completed, I will let you know how well this works. Based on your indications I expect success. Thanks again.

jhb50 · 11-28-2015, 03:21 PM

I have counted on the following as stated above and elsewhere to ensure that when my .epub books are sent to my android device, that any updated description contained in the caliber.db and .opf is inserted in the exported .epub .

Quote:

When you update metadata in Calibre it only updates it in the Calibre metadata database - it is not embedded into the ePub file at that point. Calibre uses a 'lazy update' technique for performance reasons to make updating metadata faster as database updates are quick.
It is only updated in the ePub file if you do a conversion to ePub; Use Send to Device; or use Save to Disk (to export the file from Calibre).

I have numerous examples where the original .epub has a blank "description" and the .opf .db has an updated one. Conversion always updates the new .epubs correctly, but "Send to" and "Save to" update the exported .epub for some titles, but do not update the exported .epub for others. Hence my android ends up with numerous books with blank descriptions even though I have updated them on calibre.

I can report this as a bug with supporting data but thought it best to post here first.
I am using 2.45 and the preferences for Send to and Save to are set to the default which includes "update metadata in saved copies".

eschwartz · 11-28-2015, 09:17 PM

The books should definitely have the updated metadata.
By which I mean, no one else that I know of has ever had that problem, and had it be calibre's fault.

Is it possible you resent an already-sent book and your ereader app is caching the old metadata (which doesn't have the description)?

eschwartz · 11-28-2015, 09:20 PM

Also note that the above statements are no longer true; there is now another way to update the metadata.

More recent versions of calibre include an "Embed Metadata" tool (add it from Preferences ==> Toolbar).

Nevertheless, as I said, send-to-device will still update the metadata.

jhb50 · 11-29-2015, 03:15 AM

I agree that there are a number of ways to force the metadata but I would like it to update automatically on "Save to" and "Send to".

That said I have diagnosed the problem and determined that when the embedded description in the .opf section of a source .epub is of the form

Quote:

<description xmlns="http://purl.org/dc/elements/1.1/"><The must-have ....

rather than

Quote:

<dc:description><The must-have ....

, caliber will correctly create its .opf file containing <dc:description><The must-have .... however it will not correct the .opf section in the exported .epub file.

I think this qualifies as a bug since calibre correctly identifies the epub/opf <description and corrects it to <dc:description in its own .opf but not in the exported epub.

However, I also note that the epub viewer also fails to load these <description(s) into its Comments box, whereas it does load <dc:description(s) correctly, and Moon Reader also fails to accept <description as a valid description, so I conclude the real problem is with the use of <description in these epub source files over which I have no control, however I would expect Calibre to output the valid <dc:description format from its corrected .opf in the exported epubs.

In support of this request, I note that if the <dc:description field in a source epub is changed, that the change is not only reflected in the caliber .opf file but that the .opf content is also used to update the .opf in the exported epub. Hence the mechanism exists to reflect the <dc:description label change as well in the exported epub.

I have many downloaded source epubs that utilize <description rather than <dc:description so its not a minor requirement for me.

Should I raise a bug report?

eschwartz · 11-29-2015, 03:33 AM

I think that is a legitimate bug, yes.
Good thing you found out the reason.

(At the very least, you shouldn't worry about reporting it and getting laughed at, because it most certainly isn't a silly-non-bug.

)

jhb50 · 11-29-2015, 03:37 AM

Tomorrows project..Learn how to report a bug :-)
Thanks for your quick response and support.

eschwartz · 11-29-2015, 03:46 AM

http://calibre-ebook.com/bugs

jhb50 · 11-30-2015, 02:58 AM

I have determined that the use of <meta <description is valid EPUB syntax and checks out fine using the EPUB3 validity checker http://validator.idpf.org/ and displays correctly within caliber and the epub-metadata-editor. The bug is within Moon Reader and I have reported its failure to display these descriptions to the author. In the meantime any such epubs can be converted epub to epub which will generate <dc:description syntax which Moon Reader displays correctly.

eschwartz · 11-30-2015, 07:58 AM

But you said it isn't recognized by the ebook-viewer, and also that calibre doesn't update the <description>?

And I still think it is calibre's job to emit metadata in the most compatible way.

jhb50 · 11-30-2015, 03:07 PM

Quote:

Originally Posted by eschwartz

But you said it isn't recognized by the ebook-viewer

I was referring to epub-metadata-editor https://github.com/benchen71/epub-metadata-editor but the current release does correctly display the <meta <description

Quote:

and also that calibre doesn't update the <description>?

It does not change <meta <description to <dc:description> as it does on a conversion. I will point this out in a bug report but its up to Koval. See #1521324 Exported EPUB not changing <meta <description to <dc:description as does EPUB-EPUB conversion

Quote:

And I still think it is calibre's job to emit metadata in the most compatible way.

I agree but I can live with a MoonReader fix.

jhb50 · 12-01-2015, 01:16 AM

Kovid has responded to this with the following, which establishes the difference between conversions and exports. While the metadata will be updated in both EPUB's, only conversions will see empty namespace prefixes updated to dc namespace prefixes.

With respect to Moon Reader, the author will fix the display of non prefix description namespaces in the next release (3.3.4?)

"Not using a dc namespace prefix is perfectly valid. In XML namespace
prefixes are unimportant as long as the actual namespace value they map
to is correct. In this case your EPUB file is choosing to use the empty
namespace prefix, via an extra xmlns declaration.

While calibre could theoretically check every single tag in the opf to
see if it uses the "commonly used" xml namespace prefix, and then
replace the tag if it does not, that is an expensive operation, and also
potentially destructive. After all the user might very well have their
own reasons for choosing different namespace prefixes. As such it is not
suitable for metadata updates. Conversions on the other hand are
designed to fix as many such possible compatibility issues as they can,
to end up with a file that works on the widest possible array of reading
devices."

10-11-2013, 05:21 PM	#16
eurasiarc Member Posts: 13 Karma: 10 Join Date: Sep 2013 Device: Nook HD +	Montano Not Showing Updated Epub Metadata I just used Calibre to edit some tags and reconverted the books to epub format. The tags for these books appear in the book information in the Calibre reader for PC. Then I used Calibre Companion to sync the metadata and also send a new epub book that contains the tags I added. I can see the new tags and metadata for the book listings in Calibre companion, but none of the edited metadata are tags appear in the book information in the Montano library, which is set to search for books in the same folder on the external sdcard that is used to send books via Calibre Companion. Strangely some of the epub books in the Montano Library, which are exactly the same books in the Calibre Companion library have tags I have never seen before. A few books have as many as twenty or thirty tags, but none of the tags for epub books that I created using calibre are listed in the tags list in Montano. Any ideas about what could be causing this? Even the new epub book with tags that appear in Calibre and Calbre companion appears without any tags in Montano reader.

10-11-2013, 06:21 PM	#18
Adoby Handy Elephant Posts: 1,736 Karma: 26785668 Join Date: Dec 2009 Location: Southern Sweden, far out in the quiet woods Device: Thinkpad E595, Ubuntu Mate, Huawei Mediapad 5, Bouye Likebook Plus	As I stated earlier you have to delete and reimport books in Mantano to make updated metadata show up. There is no refresh metadata function in Mantano. CC gets the information about the tags from calibre, not from inside the books. Mantano gets the information about the tags from inside the books, not from calibre. Refresh metadata in Mantano would have been a great feature. Perhaps even that Mantano check the timestamp of the books, in the background and automatically refresh metadata for updated books, and delete removed books and add new ones. Or even that Mantano detects and use a database with information saved by calibre in the top of the book folder. And rules for how the books should be added to different collections would also be nice. For instance you could have regular regular expressions and logical operators that map to a collection: sci(ence)\s?fi(ction)* OR space\s?opera* maps the book to the collection "Science Fiction", if any tag match. Last edited by Adoby; 10-11-2013 at 06:35 PM.

11-28-2015, 09:17 PM	#21
eschwartz Ex-Helpdesk Junkie Posts: 19,422 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	The books should definitely have the updated metadata. By which I mean, no one else that I know of has ever had that problem, and had it be calibre's fault. Is it possible you resent an already-sent book and your ereader app is caching the old metadata (which doesn't have the description)? Last edited by eschwartz; 11-28-2015 at 09:21 PM.

11-29-2015, 03:33 AM	#24
eschwartz Ex-Helpdesk Junkie Posts: 19,422 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	I think that is a legitimate bug, yes. Good thing you found out the reason. (At the very least, you shouldn't worry about reporting it and getting laughed at, because it most certainly isn't a silly-non-bug. ) Last edited by eschwartz; 11-29-2015 at 03:36 AM.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Calibre ebook-meta can't empty metadata	Budfudder	Library Management	7	03-12-2013 06:30 PM
How can I remove Published Date from metadata using ebook-meta	Pixelastic	Library Management	6	11-10-2012 05:11 AM
Calibre/ePub Metadata	carmenchu	Library Management	5	09-07-2011 03:13 PM
Metadata from calibre converted epub	wannabee	Kindle Formats	0	03-01-2011 12:05 AM
Can't change metadata with ebook-meta	mumdigau	Calibre	14	08-05-2010 08:58 PM

10-11-2013, 05:59 PM	#17
chaley Grand Sorcerer Posts: 11,742 Karma: 6997045 Join Date: Jan 2010 Location: Notts, England Device: Kobo Libra 2	My guess is that mantano is doing some kind of import when it first sees a book, and it's not reimporting the book when it changes. Is there a function in mantano to force a database refresh? If not, then you might be forced to delete and reimport the book. BTW: it is exactly this sort of problem that pushed us to make CC into a full fledged book manager with searching and grouping. Trying to understand all the ways that the reader apps manage metadata in their libraries was a lost cause.

10-12-2013, 01:58 PM	#19
eurasiarc Member Posts: 13 Karma: 10 Join Date: Sep 2013 Device: Nook HD +	Thanks for the latest information and suggestions on editing metadata in Calibre and getting the new metadata to appear in readers such as Mantano. I have been busy making changes in the tags for many epub books in my library before sending them to my device and deleting and reimporting them into Mantano to see if the new tags appear. (By deleting, I mean deleting the database entries for Mantano reader, not the books themselves.) When I get this completed, I will let you know how well this works. Based on your indications I expect success. Thanks again.

11-28-2015, 09:20 PM	#22
eschwartz Ex-Helpdesk Junkie Posts: 19,422 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	Also note that the above statements are no longer true; there is now another way to update the metadata. More recent versions of calibre include an "Embed Metadata" tool (add it from Preferences ==> Toolbar). Nevertheless, as I said, send-to-device will still update the metadata.

11-29-2015, 03:37 AM	#25
jhb50 Member Posts: 10 Karma: 10 Join Date: Sep 2011 Device: none	Tomorrows project..Learn how to report a bug :-) Thanks for your quick response and support.

11-29-2015, 03:46 AM	#26
eschwartz Ex-Helpdesk Junkie Posts: 19,422 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	http://calibre-ebook.com/bugs

11-30-2015, 02:58 AM	#27
jhb50 Member Posts: 10 Karma: 10 Join Date: Sep 2011 Device: none	I have determined that the use of <meta <description is valid EPUB syntax and checks out fine using the EPUB3 validity checker http://validator.idpf.org/ and displays correctly within caliber and the epub-metadata-editor. The bug is within Moon Reader and I have reported its failure to display these descriptions to the author. In the meantime any such epubs can be converted epub to epub which will generate <dc:description syntax which Moon Reader displays correctly.

11-30-2015, 07:58 AM	#28
eschwartz Ex-Helpdesk Junkie Posts: 19,422 Karma: 85397180 Join Date: Nov 2012 Location: The Beaten Path, USA, Roundworld, This Side of Infinity Device: Kindle Touch fw5.3.7 (Wifi only)	But you said it isn't recognized by the ebook-viewer, and also that calibre doesn't update the <description>? And I still think it is calibre's job to emit metadata in the most compatible way.

12-01-2015, 01:16 AM	#30
jhb50 Member Posts: 10 Karma: 10 Join Date: Sep 2011 Device: none	Kovid has responded to this with the following, which establishes the difference between conversions and exports. While the metadata will be updated in both EPUB's, only conversions will see empty namespace prefixes updated to dc namespace prefixes. With respect to Moon Reader, the author will fix the display of non prefix description namespaces in the next release (3.3.4?) "Not using a dc namespace prefix is perfectly valid. In XML namespace prefixes are unimportant as long as the actual namespace value they map to is correct. In this case your EPUB file is choosing to use the empty namespace prefix, via an extra xmlns declaration. While calibre could theoretically check every single tag in the opf to see if it uses the "commonly used" xml namespace prefix, and then replace the tag if it does not, that is an expensive operation, and also potentially destructive. After all the user might very well have their own reasons for choosing different namespace prefixes. As such it is not suitable for metadata updates. Conversions on the other hand are designed to fix as many such possible compatibility issues as they can, to end up with a file that works on the widest possible array of reading devices."

Advert

Advert