Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 07-12-2018, 05:02 PM   #1
citronalco
Member
citronalco began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Jul 2018
Device: PocketBook Touch HD
[Metadata Source Plugin] Deutsche Nationalbibliothek

This plugin uses the catalogue of the Deutsche Nationalbibliothek (DNB) to retrieve metadata.
DNB is the German central archival library. German publishers are required to send them a copy of every book for archival, so it's the largest metadata source for literature published in Germany.

Main Features
This plugin supports retrieval of all relevant metadata (authors, title, cover, ISBN, edition, tags, publication date, languages, publisher, comments, series, series index, URN and DNB-IDN).
For books without series information it can try to extract series and series index from the title. Standardized GND subjects and/or non-GND subjects can be used as tags.
This plugin works with Python 2 and Python 3. You don't need a SRU Access Token any more.

Limitations
The returned publication date contains only the year, not the precise date.

Installation
Download the attached zip file and install the plugin as described in the Introduction to plugins thread.
The plugin is also available in Calibre's Plugin Updater.

How to report Bugs
If you find any issues please report them in this thread or on Github.

Version History
Spoiler:

v2.0.6 - 11 Jul 2018
  • First version posted on this forum

v2.0.7 - 5 Oct 2018
  • Add some error reporting

v2.1.0 - 4 Jan 2019
  • Fix retrieval of publication date and publisher name for new Calibre versions
  • Improve series extraction
  • Code cleanup

v2.1.1 - 5 Jan 2019
  • Further improve series extraction

v2.1.2 - 7 Oct 2019
  • Fix retrieval by command line
  • Tidy up query creation, at least a bit
  • Workaround for SRU issue with ? and + signs
  • Improve retrieval of series info from metadata
  • Improve series guesser

v2.1.3 - 17 May 2020
  • Fix regression when searching without author
  • Improve series detection
  • Preliminary support for Python 3

v3.0.0 - 9 Jul 2020
  • Full support for Python 2 and Python 3

v3.0.1 - 29 March 2021
  • Remove web scraping and SRU token: Beginning of April 1st 2021 DNB's SRU interface is publicly available
  • If ISBN is given only search for the ISBN and ignore title and authors. This is how other plugins handle this, too
  • Improve XPATH code
  • Fix duplicate "urn:urn:" prefix at URN
v3.1.1 - 10 April 2021
  • Improve logging
  • Fix language extraction
  • Reorder queries to improve results quality
  • Fix filtering out audio books
  • Fix multiline publisher location
  • Improve creation of title variants for queries
  • Improve handling of titles with numbers with leading zeros
  • Don't query DNB for more results when good results are already found
  • Update minimum required Calibre version to 0.9.33
v3.1.2 - 10 May 2021
  • Fix regression in series extraction
  • Fix skipping of audio books
  • Minor improvements
Attached Files
File Type: zip dnb-metadata-plugin-3.1.2.zip (25.9 KB, 3771 views)

Last edited by citronalco; 05-10-2021 at 05:00 PM. Reason: New plugin version released
citronalco is offline   Reply With Quote
Old 10-02-2018, 06:39 AM   #2
Lawer
Junior Member
Lawer began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Oct 2018
Device: none
Hello! This plugin sounds so good.

Register at Deutsche Nationalbibliothek was so easy.

But i´m not sure, what kind of sru access token is necessary.

What kind of format ? Marc 21 or RDF ? What meens that?
What kind of archive I need ? With or without GND???

I´m really helpless!
Lawer is offline   Reply With Quote
Advert
Old 10-02-2018, 08:58 AM   #3
Lawer
Junior Member
Lawer began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Oct 2018
Device: none
Finally i´ve got the sru Access Token!

I wrote the sru Access token Code in the Plugin - but it didn´t work.

No Metadatas found.

Without the sru Access token Code it works!

What´s wrong?
Lawer is offline   Reply With Quote
Old 10-05-2018, 11:09 AM   #4
citronalco
Member
citronalco began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Jul 2018
Device: PocketBook Touch HD
First of all, Lawer's issue with the non working SRU token was a non-issue, turned out to be a copy-paste problem.

The description of the steps required to get a SRU Access Token was a little too vague:
When requesting the token tell them you want to have access to the title data catalogue ("Titeldaten-Katalog") in the format MARC21-XML. I have updated the README file accordingly.

Marc21, Marc21-XML and RDF are different data formats, just like JPG and PNG.
DNB offers several catalogues, the Titeldaten-Katalog contains book information (title, author, publication date,...), the GND ("Gemeinsame Normdatei") contains e.g. biographic information about authors, valid descriptive tags etc.
citronalco is offline   Reply With Quote
Old 05-22-2020, 05:31 PM   #5
citronalco
Member
citronalco began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Jul 2018
Device: PocketBook Touch HD
In the meantime (2018) I've updated the plugin a few times and fixed a few bugs some helpful guys reported to me.
It's now also available in a Python 3 variant: https://github.com/citronalco/calibre-dnb/releases

Maybe I should note that the DNB is the German central archival library. Every publisher is required to hand them a book over for archival. So it's the largest metadata source for German literature.

Last edited by citronalco; 05-22-2020 at 05:36 PM.
citronalco is offline   Reply With Quote
Advert
Old 05-26-2020, 05:18 AM   #6
collaco
Junior Member
collaco began at the beginning.
 
collaco's Avatar
 
Posts: 1
Karma: 10
Join Date: May 2020
Device: iPad
Just installed it - got my access token from DNB in less then 48h - works perfect! Thanx
collaco is offline   Reply With Quote
Old 07-09-2020, 12:49 PM   #7
citronalco
Member
citronalco began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Jul 2018
Device: PocketBook Touch HD
New release: v3.0.0

The plugin now supports Python 2 and Python 3. No different ZIP files required.

Fixes:
- Fixed downloading metadata in Python 3 without SRU.

You can download the newest release from here:
https://github.com/citronalco/calibre-dnb/releases

If you find any issues please report them on Github.
citronalco is offline   Reply With Quote
Old 07-09-2020, 05:18 PM   #8
citronalco
Member
citronalco began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Jul 2018
Device: PocketBook Touch HD
Does anyone know how I get my plugin listed on the plugins page (https://plugins.calibre-ebook.com/)?
citronalco is offline   Reply With Quote
Old 07-09-2020, 06:33 PM   #9
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 17,350
Karma: 20292561
Join Date: Mar 2012
Location: Sydney Australia
Device: none
Read the Notes for developers at the top of the Index of plugins thread.

You'll need to edit the opening post of this thread so that conforms to the layout and content of other plugin threads - use one of the other Metadata Download plugin's first post as a guide - main thing is that the plugin zip must be attached to the first post.

The next release of calibre will be released soon. It requires adjustments to be made to plugin code to make them python 3 compatible, see the relevant sticky threads at the top of this forum for details.

When you're ready to roll, PM me or one of the other mods with the details you want in the index.

BR
BetterRed is offline   Reply With Quote
Old 07-10-2020, 07:00 PM   #10
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 17,350
Karma: 20292561
Join Date: Mar 2012
Location: Sydney Australia
Device: none
@citronalco - you should have edit permission now.

Paul has attached the plugin to the first post for you, but you probably want to add a couple of items, like Version History etc. And maybe update the plugin to be Python 3 compliant.

Could you remove or move the link to Gitub in the first post, it could confuse users who are accustomed to getting their plugins from here or via calibre itself.

I see you've done the P3 changes - I missed that.

Send me a PM with the details to put in the index when you're ready..

BR

Last edited by BetterRed; 07-10-2020 at 07:11 PM.
BetterRed is offline   Reply With Quote
Old 02-26-2021, 02:09 AM   #11
beedaddy
Connoisseur
beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!
 
Posts: 84
Karma: 100964
Join Date: Oct 2011
Location: Germany (south-west)
Device: Tolino Vision 5, PocketBook Touch HD 3
@citronalco I certainly did something wrong and I hope you can give me a hint: The DNB gave me a access token, but fetching metadata only returns me a cover image but no other metadata like comments or so. What could be the reason for that? (And I wonder what I should select for "fetch subjects"…)
beedaddy is offline   Reply With Quote
Old 03-05-2021, 05:08 AM   #12
citronalco
Member
citronalco began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Jul 2018
Device: PocketBook Touch HD
Quote:
Originally Posted by beedaddy View Post
@citronalco I certainly did something wrong and I hope you can give me a hint: The DNB gave me a access token, but fetching metadata only returns me a cover image but no other metadata like comments or so. What could be the reason for that? (And I wonder what I should select for "fetch subjects"…)
There are several things you could check:
- In the generic metadata configuration dialogue (the window where you can select which metadata plugin you want to use): Have you selected any metadata fields to download?
- The plugin returns the same metadata, albeit much slower, without a token (token field empty in the config window). Do you get metadata without a token? If yes: Check if you've entered the token correctly (maybe you accidentally pasted a space at the end or similar)
- When retrieving metadata has finished you could have a look at the protocol and search for the "Query URL". You can simply paste the URL into a browser and check if you get a longish XML file or a error message.

If nothing of the above solves your problem: Please send me the protocol of a search.

Just in this moment DNB seems to have issues with their API anyway: Currently they don't check the token at all and accept anything - Maybe they simply have issues with their infrastructure. I doubt they intentionally opened their API for unauthenticated users - though I'd be happy if they would.

About the subjects:
There are two types of subjects:
- GND subjects are selected by librarians from a standardized catalogue (the "Gemeinsame NormDatei"), so they are are not available for every book, but they are usually high quality
- Non GND subjects are simply delivered by the publisher and can be anything. There's no quality checking done by DNB, so it can be something useful or just marketing SEO crap.
Older books usually do not have non-GND but GND subjects, with newer books it's the other way round.
Personally I prefer the setting "GND subjects if available, otherwise non-GND subjects", but it's a matter of taste.

Last edited by citronalco; 03-05-2021 at 05:12 AM.
citronalco is offline   Reply With Quote
Old 03-08-2021, 11:03 AM   #13
Henning
Member
Henning began at the beginning.
 
Posts: 23
Karma: 10
Join Date: Mar 2012
Location: Berlin
Device: iPad
@citronalco

For your information:
Email from "Schnittstelle-Service:
BITTE BEACHTEN SIE: Zukünftig wird die Nutzung der SRU-Schnittstelle ohne vorherige Registrierung möglich sein. Die technische Umstellung ist Anfang März bereits erfolgt, so dass Sie jetzt schon auf die gewünschte Schnittstelle zugreifen können. Die vollständige Umstellung wird am 1. April vollzogen sein und dann auch auf der Webseite entsprechend kommuniziert, Sie können die Schnittstelle aber ab sofort nutzen und müssen diesbezüglich nichts mehr tun.

Wir informieren regelmäßig über wichtige Änderungen und Neuerungen an den Schnittstellen der Deutschen Nationalbibliothek (DNB). Ebenso halten wir Sie zeitnah über Störungen und Wartungsarbeiten auf dem Laufenden.
Wenn Sie diesen Service nutzen möchten, melden Sie sich bitte für den Mailverteiler an.*
SRU-Schnittstelle: https://lists.dnb.de/mailman/listinfo/sru-schnittstelle


==> as far as I understand: the token-system is outdated - Your wish to establish a system without registration (and token) has come thru.
Henning is offline   Reply With Quote
Old 03-10-2021, 01:36 AM   #14
beedaddy
Connoisseur
beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!beedaddy rocks like Gibraltar!
 
Posts: 84
Karma: 100964
Join Date: Oct 2011
Location: Germany (south-west)
Device: Tolino Vision 5, PocketBook Touch HD 3
Thanks @citronalco for your detailed explanation. I will double-check the settings. I think that sometimes DNB simply lacks the blurb.
beedaddy is offline   Reply With Quote
Old 03-10-2021, 06:12 PM   #15
citronalco
Member
citronalco began at the beginning.
 
Posts: 11
Karma: 10
Join Date: Jul 2018
Device: PocketBook Touch HD
@Henning: Good news, thanks!
citronalco is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[Metadata Source Plugin] Comicvine chewt0y Plugins 62 04-03-2020 12:54 AM
[Metadata Source Plugin] INMONDADORI Pr.BarnArt Plugins 5 07-29-2016 05:56 AM
[Metadata Source Plugin] Amazon.CN fated Plugins 0 11-20-2014 04:59 PM
[Metadata Source Plugin] DIZZIE_NL Pr.BarnArt Plugins 7 08-11-2014 01:48 PM
[Metadata Source Plugin] catawiki drMerry Plugins 0 06-28-2011 08:42 PM


All times are GMT -4. The time now is 08:13 AM.


MobileRead.com is a privately owned, operated and funded community.