[Metadata Source Plugin] FictionDB

[kiwidude]

This plugin was requested on the Plugin Ideas sticky thread, and allows retrieving book metadata from the www.fictiondb.com website.

This website can have good descriptions for books and often excellent potential tags data with a range of genre, sub-genre, time period and tags information that can be optionally retrieved into your tags column (configure in the preferences for this plugin).

Note also that while the website supports multiple types of series (not only an author series but also multi-author and publisher series), this plugin will only retrieve for author series, and only if an order has been defined. This is due to the limitations of the calibre series column.

Main Features:

• Can retrieve title, author, comments, series, publisher, publication date, rating, tags and (small!) cover
• Configure which of the genres, sub-genres, themes, time period and tags sections you want retrieved into your calibre tags column.
• Also retrieves a FictionDB id, which can be used to directly jump to the page for a specific book from the book details pane (like Goodreads/ISBN etc).

Special Notes:

• Help Documentation
• Change History
• Requires Calibre 2.85.1 or later.

Paypal Donations:

• If you find this or any of my other plugins useful please feel free to show your appreciation. I have spent many hundreds of unpaid hours in their development and support so any encouragement for me to continue is appreciated!
  

[copyrite]

Am trying to use this in Calibre v.0.8.42 to poll for metadata with no luck. Under configuration all boxes are checked. I've even checked some of the books using the Search the Internet plug-in (FictionDB for book) and the info is definitely there. Any suggestions?

[kiwidude]

@copyrite - I'm afraid I can't verify anything without specific details

Please post the log output (in a spoiler) from the metadata download window, that has all the details I need of the title/author/isbn and what it did succeed in doing or any errors that were thrown.

[copyrite]

Here ya go...

Spoiler:

Starting job: Download metadata for 1 books
Running identify query with parameters:
{u'authors': [u'Anne Mather'], u'identifiers': {u'amazon': u'0373631529', u'google': u'Z6wuAAAACAAJ', u'isbn': u'9780263805727'}, u'timeout': 30, u'title': u'The Shrouded Web'}
Using plugins: FictionDB
The log from individual plugins is below

****************************** FictionDB ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3')]
Found 0 results
Downloading from FictionDB took 0.703999996185
Querying: http://www.fictiondb.com/search/searchresults.htm - srchtxt=9780263805727&styp=4
ISBN match location: u'http://www.fictiondb.com/author/anne-mather~the-shrouded-web~25243~b.htm'
FictionDB book url: u'http://www.fictiondb.com/author/anne-mather~the-shrouded-web~25243~b.htm'
get_details failed for url: u'http://www.fictiondb.com/author/anne-mather~the-shrouded-web~25243~b.htm'
Traceback (most recent call last):
File "calibre_plugins.fictiondb.worker", line 37, in run
File "calibre_plugins.fictiondb.worker", line 61, in get_details
IOError: [Errno 13] Permission denied: u'f:\\fictiondb.html'

************************************************** ******************************
The identify phase took 0.99 seconds
The longest time (0.704000) was taken by: FictionDB
Merging results from different sources and finding earliest publication dates from the xisbn service
We have 0 merged results, merging took: 0.00 seconds
Failed to download metadata for The Shrouded Web

****************************** FictionDB Covers ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.3 (KHTML, like Gecko) Chrome/6.0.472.63 Safari/534.3')]
Failed to download valid cover
Took 0.679000139236 seconds
No cached cover found, running identify
Querying: http://www.fictiondb.com/search/searchresults.htm - srchtxt=9780263805727&styp=4
ISBN match location: u'http://www.fictiondb.com/author/anne-mather~the-shrouded-web~25243~b.htm'
FictionDB book url: u'http://www.fictiondb.com/author/anne-mather~the-shrouded-web~25243~b.htm'
get_details failed for url: u'http://www.fictiondb.com/author/anne-mather~the-shrouded-web~25243~b.htm'
Traceback (most recent call last):
File "calibre_plugins.fictiondb.worker", line 37, in run
File "calibre_plugins.fictiondb.worker", line 61, in get_details
IOError: [Errno 13] Permission denied: u'f:\\fictiondb.html'

No cover found

************************************************** ******************************
Download complete, with 1 failures

Does it somehow need to query my account information?

Thanks!

[kiwidude]

Thanks for that. Yuck - that looks like a line of debug code got left in my official release - ugh. I will take a look when I get home and push up a new release.

[kiwidude]

Changes in this release:

• Remove debug code incorrectly left in initial release

@copyrite - give this a whirl, hopefully that should fix the issue, Surprised no-one else noticed - they must all have F-drives :-)

Thanks for reporting it.

[copyrite]

Alas, I only go to "D".

But you, sir, get an A PLUS!

(and a few pennies, to boot )

Thanks so much for the lightning fast response!

[DedTV]

Looks like this isn't working anymore. Returns a no matches found error on everything. Although the URLs in the log it picks up from the search result seem fine and go to the right book when I put them in a browser.

Here's a log:

Spoiler:

****************************** FictionDB ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.78 Safari/532.5')]
Found 0 results
Downloading from FictionDB took 1.4849998951
Querying: http://www.fictiondb.com/search/searchresults.htm - srchtxt=9780747573616&styp=4
ISBN match location: u'http://www.fictiondb.com/author/jk-rowling~harry-potter-and-the-chamber-of-secrets~134192~b.htm'
FictionDB book url: u'http://www.fictiondb.com/author/jk-rowling~harry-potter-and-the-chamber-of-secrets~134192~b.htm'
Could not find title/authors/fictiondb id for u'http://www.fictiondb.com/author/jk-rowling~harry-potter-and-the-chamber-of-secrets~134192~b.htm'
FictionDB: u'jk-rowling~harry-potter-and-the-chamber-of-secrets~134192~b' Title: None Authors: None

[kiwidude]

That's because they have completely revamped their website, which always breaks metadata download plugins. I will have to see if the plugin can be resurrected...

[DedTV]

Originally Posted by kiwidude

That's because they have completely revamped their website, which always breaks metadata download plugins. I will have to see if the plugin can be resurrected...

Cool. Sent ya a cup of coffee for giving it a look.
It's the main source I use for book synopsis, series info and genre tags so it'd be nice to have it working again.

[kiwidude]

Changes in this release:

• Updated for FictionDB website revamp
• Add option for retrieving Themes section as tags (off by default)

[DedTV]

Originally Posted by kiwidude

Changes in this release:

• Updated for FictionDB website revamp
• Add option for retrieving Themes section as tags (off by default)

Dayum that was fast.
Much thanks!

[ilovejedd]

Hmm, plug-in doesn't seem to want to work if the book title has an apostrophe.

FictionDB page:
The Halfling's Gem
Charon's Claw

calibre, version 0.8.70
Download complete: <p>Finished downloading metadata for <b>2 book(s)</b>. Proceed with updating the metadata in your library?<p>Could not download metadata and/or covers for 2 of the books. Click "Show details" to see which books.

The Halfling's Gem (Failed metadata) (Failed cover)
Charon's Claw (Failed metadata) (Failed cover)

#################### Log for Charon's Claw ####################
Running identify query with parameters: 
{u'authors': [u'R.A. Salvatore'], u'identifiers': {}, u'timeout': 30, u'title': u"Charon's Claw"} 
Using plugins: FictionDB 
The log from individual plugins is below 

****************************** FictionDB ****************************** 
Request extra headers: [('User-agent', 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)')] 
Found 0 results 
Downloading from FictionDB took 0.371000051498 
Querying: http://www.fictiondb.com/search/searchresults.htm - status=-&rating=-&pubgroup=0&isbn=&title=Charon%2527s+Claw&list=-&author=Salvatore&genretype=--&styp=6&datepublished=&imprint=0&srchtxt=multi&series=&myrating=-&dateread=&sgcode=0&tpcode=0&published=
No matches found with query: u'http://www.fictiondb.com/search/searchresults.htm' - 'status=-&rating=-&pubgroup=0&isbn=&title=Charon%2527s+Claw&list=-&author=Salvatore&genretype=--&styp=6&datepublished=&imprint=0&srchtxt=multi&series=&myrating=-&dateread=&sgcode=0&tpcode=0&published=' 

******************************************************************************** 
The identify phase took 0.41 seconds 
The longest time (0.371000) was taken by: FictionDB 
Merging results from different sources and finding earliest publication dates from the xisbn service 
We have 0 merged results, merging took: 0.00 seconds 
Failed to download metadata for Charon's Claw 

****************************** FictionDB Covers ****************************** 
Request extra headers: [('User-agent', 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)')] 
Failed to download valid cover 
Took 0.240000009537 seconds 
No cached cover found, running identify
Querying: http://www.fictiondb.com/search/searchresults.htm - status=-&rating=-&pubgroup=0&isbn=&title=Charon%2527s+Claw&list=-&author=Salvatore&genretype=--&styp=6&datepublished=&imprint=0&srchtxt=multi&series=&myrating=-&dateread=&sgcode=0&tpcode=0&published=
No matches found with query: u'http://www.fictiondb.com/search/searchresults.htm' - 'status=-&rating=-&pubgroup=0&isbn=&title=Charon%2527s+Claw&list=-&author=Salvatore&genretype=--&styp=6&datepublished=&imprint=0&srchtxt=multi&series=&myrating=-&dateread=&sgcode=0&tpcode=0&published='
No cover found 

******************************************************************************** 

#################### Log for The Halfling's Gem ####################
Running identify query with parameters: 
{u'authors': [u'R.A. Salvatore'], u'identifiers': {}, u'timeout': 30, u'title': u"The Halfling's Gem"} 
Using plugins: FictionDB 
The log from individual plugins is below 

****************************** FictionDB ****************************** 
Request extra headers: [('User-agent', 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)')] 
Found 0 results 
Downloading from FictionDB took 0.311000108719 
Querying: http://www.fictiondb.com/search/searchresults.htm - status=-&rating=-&pubgroup=0&isbn=&title=The+Halfling%2527s+Gem&list=-&author=Salvatore&genretype=--&styp=6&datepublished=&imprint=0&srchtxt=multi&series=&myrating=-&dateread=&sgcode=0&tpcode=0&published=
No matches found with query: u'http://www.fictiondb.com/search/searchresults.htm' - 'status=-&rating=-&pubgroup=0&isbn=&title=The+Halfling%2527s+Gem&list=-&author=Salvatore&genretype=--&styp=6&datepublished=&imprint=0&srchtxt=multi&series=&myrating=-&dateread=&sgcode=0&tpcode=0&published=' 

******************************************************************************** 
The identify phase took 0.45 seconds 
The longest time (0.311000) was taken by: FictionDB 
Merging results from different sources and finding earliest publication dates from the xisbn service 
We have 0 merged results, merging took: 0.00 seconds 
Failed to download metadata for The Halfling's Gem 

****************************** FictionDB Covers ****************************** 
Request extra headers: [('User-agent', 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)')] 
Failed to download valid cover 
Took 0.240000009537 seconds 
No cached cover found, running identify
Querying: http://www.fictiondb.com/search/searchresults.htm - status=-&rating=-&pubgroup=0&isbn=&title=The+Halfling%2527s+Gem&list=-&author=Salvatore&genretype=--&styp=6&datepublished=&imprint=0&srchtxt=multi&series=&myrating=-&dateread=&sgcode=0&tpcode=0&published=
No matches found with query: u'http://www.fictiondb.com/search/searchresults.htm' - 'status=-&rating=-&pubgroup=0&isbn=&title=The+Halfling%2527s+Gem&list=-&author=Salvatore&genretype=--&styp=6&datepublished=&imprint=0&srchtxt=multi&series=&myrating=-&dateread=&sgcode=0&tpcode=0&published='
No cover found 

********************************************************************************

[kiwidude]

@ILoveJDD - looking at the encoding in the log you show - are you sure those are not "curly" apostrophes rather than standard ones?

[ilovejedd]

Originally Posted by kiwidude

@ILoveJDD - looking at the encoding in the log you show - are you sure those are not "curly" apostrophes rather than standard ones?

Nope. Before I downloaded metadata, I did a search and replace for curly apostrophes to regular ones. I believe %25 corresponds to the % sign so I'm guessing it's being double encoded or something. Perhaps the plugin searches for "The Halfling%27s Gem" and the website is translating it as "The Halfling%2527s Gem".