Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 02-04-2022, 02:36 PM   #1
feuille
Member
feuille began at the beginning.
 
Posts: 19
Karma: 10
Join Date: May 2020
Device: android smartphone
[Metadata Source Plugin] ISFDB3

[Metadata Source Plugin] ISFDB3 - Version 1.0.2 - 02-06-2022

Downloads metadata and covers from the Internet Speculative Fiction Database (http://www.isfdb.org/)

The ISFDB:

This database provides a lot of data for sf titles and publications (covers, artists, translations, prices, notes, ...) and references to other sources, compiled by volunteers.

A web search form is available under http://www.isfdb.org/cgi-bin/adv_search_menu.cgi
There is also a web API under http://www.isfdb.org/wiki/index.php/Web_API, but this is not in use yet by the plugin, because the interface only supplies a subset of the data.

The data model distincts between titles and publications (connected m:n) and the database has also tables for series, translations, covers, ... A dump for MySQL is available under http://www.isfdb.org/wiki/index.php/ISFDB_Downloads.

Background:

This plugin is a fork of Adrianna Pińska's ISFDB2. For more information see https://github.com/confluence/isfdb2-calibre
Adriana explained the very clever structure of her plugin herself in a YouTube video: »Custom metadata plugins for Caliber: cataloging an old paper library« (https://www.youtube.com/watch?v=UF6HAn5-YD0).

In mid 2021 I forked the codebase and make some changes and additions to the code for my needs.
Adriana contacted me and wrote:
"Hello! I haven't worked on this for a while, but I intend to get back to it (I still haven't finished cataloguing my library!).
When I do I will go through your code and see if I can merge some of the features. (...)
I don't mind if this plugin is submitted to the plugin repository -- but I am not active on the Mobileread forums, so I am unlikely to do this anytime soon.
You are welcome to! If you decide to submit your forked version, please change the name (to isfdb3-calibre?), to make it clear that it's not the same plugin and may have slightly different behaviour."

So I have created the plugin ISFDB3 and submitted it to the plugin repository at mobileread.

Main changes compared to ISFDB2:

1) Different search strategy for publications
2) No mixing of identical titles

to 1): ISFDB2 searches publications (if no id is available) with the specified title and author. Publications are only found if they can be found with the same search term as the title (i.e. have the same name).
ISFDB3 uses the list of publications in the title record and follows the links. It also finds publications that contain the title (usually a short story) but have a different name, i.e. anthologies, magazines, ...
Extreme example: H. P. Lovecraft, In the Vault. ISFDB2 finds one title and no publications, ISFDB3 95 publications.
Another example: Publications are not found by ISFDB2, if the title/publication pair as only a slightly different spelling:
Title: Best S.F. Stories from New Worlds (http://www.isfdb.org/cgi-bin/title.cgi?36317), but publication reads: Publication: Best SF Stories from New Worlds.

to 2): Calibre's default behavior merges titles and publicatications with the same author and title in one result, regardles of other dates (publication date, series, publisher, ...).
Excerpt from https://github.com/kovidgoyal/calibr...s/identify.py:
---
def merge_metadata_results(self, merge_on_identifiers=False):
'''
Merge results with identical title and authors or an identical identifier
'''
---
Therefore, to preserve all search results, ISFDB3 qualifies titles with the ISFDB id before put it in the result queue.
Example: K. H. Scheer, Expedition. The title was published six times in the years 1961 - 1980, in different series and adaptations. ISFDB2 returns only one publication, ISFDB3 all.
Other example (H. P. Lovecraft, In the Vault):
title list has two title records (in ISFDB2 there are merged to one):
title record one: http://www.isfdb.org/cgi-bin/title.cgi?2946687 -> pub record http://www.isfdb.org/cgi-bin/pl.cgi?868274
title record two: http://www.isfdb.org/cgi-bin/title.cgi?41896 -> a lot of pub records!

As a drawback, the qualifier has to be deleted manually or with a search-and-replace regex.

Other changes compared to ISFDB2:

- Changes character encoding in GET parameters (search strings) from utf-8 to iso-8859-1 to avoid none matches for non-ASCII chars (for example german umlauts). The database data itself are in utf-8.
- Adds additional identifiers found in ISFDB. Currently: DNB and OCLC/WorldCat (more on request).
- Modifies the method "clean_downloaded_metadata()": Fixcase title and author(s) only when the title language is english.
- Gets additional info about series hierarchy from the series page. Option for combine main series and series names
- Workarounds for not Calibre's float format compatible series numbers (61/62, roman numbers)
- Converts language field text in the isfdb to Calibre's language code

Planned Features / ToDo's:

There are a lot of to do's (and suggestions?).

- Notify the ISFDB Catalog ID (in pub record)
- Notify referenced webpages (in pub record)
- Find series in contents block (in pub record)
- Notify about translations in comments field

Limitations:

- Since there is no language field in publication records, only in title records, following the publication linkss in a title list may show up publications in not desired languages. However, the publications list in the title page has a button »Not displaying translations«, so some research is already meeded.

Version History:

v1.0.0 01-31-2022
- Initial release
v1.0.1 02-05-2022
- Small typo: none vs. None
v1.0.2 02-06-2022
- Parse error for dictionary LANGUAGES (moved from class to module scope)
- Typo in calling translate method

Installation:

Download the attached zip file and install the plugin as described in the Introduction to plugins thread (https://www.mobileread.com/forums/sh...d.php?t=118680).
The plugin is also available in Calibre's Plugin Updater.

How to report bugs and suggestions:

If you find any issues please report them in this thread or on Github: https://github.com/bertholdm/isfdb3-calibre.
Attached Files
File Type: zip isfdb3.zip (47.5 KB, 5087 views)

Last edited by feuille; 02-06-2022 at 01:17 PM. Reason: v1.0.2 02-06-2022
feuille is offline   Reply With Quote
Reply

Tags
isfdb, metdadata source plugin

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[Metadata Source Plugin] DatabazeKnih (cz) baghira Plugins 207 12-03-2021 03:10 PM
Read a book's metadata in a Metadata source plugin? J-H Development 2 03-30-2021 09:08 AM
[Metadata Source Plugin] Goodreads_rating Pr.BarnArt Plugins 17 02-28-2021 08:18 AM
[Metadata Source Plugin] wikidata compurandom Plugins 46 11-27-2020 11:32 PM
[Metadata Source Plugin] Empty Plugin? (Fake Identifier) mneimeyer Plugins 3 11-11-2019 08:07 PM


All times are GMT -4. The time now is 12:50 PM.


MobileRead.com is a privately owned, operated and funded community.