[Metadata Source Plugin] Fantastic Fiction

[Clarke219]

Originally Posted by davidfor

Sorry, I took it as a report of a change of behaviour especially coming just after a release was made.

Looking at the code, I was wrong about where the year came from. It is from near the top of the page which only date that seems to always be there. There are dates in the affiliates section, but, they appear to be for the edition that is linked to. There are also dates in the editions section, but, which do you use. For example, https://www.fantasticfiction.com/b/d...gley/enemy.htm was published in 1977, but there is an edition that is supposedly from 1901.

I am already getting the publisher and ISBN from the first edition listed. I probably should use the published date from that. But, I might try looking for the oldest edition with the same year as the published year. That looks like it will work for recent books, but, a lot of older books, such as https://www.fantasticfiction.com/d/g...son/dorsai.htm, don't have the original editions. In that case, just the year is best.

I don't know when I'll do this. It usually happens when I'm bored or trying not to do something else.

As you said, for most of the newer books, the month is the same for all of the editions so there isn't a problem. When they differ, I initially used the hard cover date. I have changed to the Kindle date figuring that it is the version that I have. I don't know if the order of the API listing is the same as the order shown on the Fantastic Fiction web page where hard cover appears first, but using the same info as you use for the publisher and ISBN seems reasonable.

[davidfor]

I have just updated the plugin to version 1.5.0. This is a mix of change by myself and @kiwidude. The list of changes are:

• New: Add Reduce header sizes option to replace h1,h2,h3 tags with h4 in comments (off by default). Done by @kiwidude.
• Fix: Titles/authors with apostrophes can result in not finding search matches. Done by @kiwidude.
• New: Get the publishing date from the oldest editions. Can be configured on or off.
• Change: Fix layout of configuration.
• Change: Code cleanup.

Calibre should announce the update in the next hour or so. If there are any issues with the plugin, please report them here.

[davidfor]

I have just released an update to this plugin. The change is:

• Change: Make compatible with upcoming calibre 6/Qt6.

There are no functional changes to the plugin but this should make it compatible with calibre 6 when it is released later in the year.

Calibre should announce the update in the next hour or so. If there are any issues with the plugin, please report them here.

[kiwidude]

Release Notes:
https://github.com/kiwidude68/calibr...fiction-v1.6.0

[kiwidude]

Release Notes
https://github.com/kiwidude68/calibr...fiction-v1.6.1

[kiwidude]

Release Notes
https://github.com/kiwidude68/calibr...fiction-v1.6.2

Catering for either website changes or variations in books - giving up on trying to get a published date from FF (other than the year to which I can only assume a date of Jan 1st). It would have been nice to parse the dates from the "Buy from Amazon" section but that data is loaded later than what is present in the html I am scrpaing. Stick to other sources like Goodreads for a more reliable published date than FF.

[nurbles62]

Today I was searching for meta data for "Seeing Red" by Nick Thacker and for some reason, none of the plugins can find it, though it is easily found on FantasticFiction, Amazon, GoodReads and other sites. Here's the relevant log section for the FF plugin:

****************************** Fantastic Fiction (1, 6, 2) ****************************** 
Found 0 results 
Downloading from Fantastic Fiction took 0.2872316837310791 
Querying: https://www.fantasticfiction.com/dbs/books2?q.parser=structured&q=%28and+db%3A%27FF%27+%27Seeing+Red%27%29&size=20&start=0&sort=visits_uk+desc&return=booktype%2Ctitle%2Catitle%2Cvtitle%2Cyear%2Cpfn%2Chasimage%2Cauthorsinfo%2Cseriesinfo%2Cdb%2Cimageloc
JSON Result: b'{"status":{"rid":"8rGMy4wxtMVmCtRHvA==","time-ms":1},"hits":{"found":51,"start":0,"hit":[{"id":"w651715","fields":{"booktype":"1","title":"Seeing Red","pfn":"b/sandra-brown/seeing-red.htm","year":"2017","authorsinfo":"b/sandra-brown|Sandra Brown|14181|FF","db":["FF"]}},{"id":"w1007418","fields":{"booktype":"1","title":"Seeing Red","pfn":"g/hannah-gray/seeing-red.htm","year":"2020","authorsinfo":"g/hannah-gray|Hannah Gray|114264|FF","seriesinfo":"NE University|2","db":["FF"]}},{"id":"w531843","fields":{"booktype":"1","title":"Seeing Red","pfn":"j/patty-jansen/seeing-red.htm","year":"2013","authorsinfo":"j/patty-jansen|Patty Jansen|67708|FF","seriesinfo":"Ambassador|1","db":["FF"]}},{"id":"w380636","fields":{"booktype":"22","title":"Aftershock","atitle":"Penance / After the Lightning / Seeing Red","pfn":"c/debra-cowan/aftershock.htm","year":"2008","authorsinfo":"c/debra-cowan|Debra Cowan|19733|FF^h/janis-reams-hudson|Janis Reams Hudson|18662|FF^s/sharon-sala|Sharon Sala|17666|FF","imageloc":"c4/c22252.jpg","db":["FF"]}},{"id":"w712638","fields":{"booktype":"1","title":"Seeing Red","pfn":"t/nick-thacker/seeing-red.htm","year":"2018","authorsinfo":"t/nick-thacker|Nick Thacker|81918|FF","seriesinfo":"Gareth Red|1","db":["FF"]}},{"id":"w452129","fields":{"booktype":"8","title":"Permanent Red","atitle":"Essays in Seeing","pfn":"b/john-berger/permanent-red.htm","year":"1960","authorsinfo":"b/john-berger|John Berger|18962|FF","imageloc":"x1/x7248.jpg","db":["FF"]}},{"id":"w140730","fields":{"booktype":"1","title":"Seeing Red","pfn":"s/jill-shalvis/seeing-red.htm","year":"2005","authorsinfo":"s/jill-shalvis|Jill Shalvis|17753|FF","seriesinfo":"Firefighters [Jill Shalvis]|3","db":["FF"]}},{"id":"w155593","fields":{"booktype":"1","title":"Ironsides Sees Red","pfn":"b/edwy-searles-brooks/ironsides-sees-red.htm","year":"1943","authorsinfo":"|Victor Gunn|22208|FF","seriesinfo":"Ironsides Cromwell|","imageloc":"n32/n160076.jpg","db":["FF"]}},{"id":"w538256","fields":{"booktype":"8","title":"Seeing Red","atitle":"Revlon and Me: A Memoir","pfn":"f/betty-ferm/seeing-red.htm","year":"2014","authorsinfo":"f/betty-ferm|Betty Ferm|9087|FF","db":["FF"]}},{"id":"w1133585","fields":{"booktype":"1","title":"Seeing Red","pfn":"b/alex-beecroft/seeing-red.htm","year":"2019","authorsinfo":"b/alex-beecroft|Alex Beecroft|55022|FF","seriesinfo":"Trowchester Blues|","db":["FF"]}},{"id":"w1111789","fields":{"booktype":"1","title":"Seeing Red","pfn":"r/stephanie-renee/seeing-red.htm","year":"2020","authorsinfo":"r/stephanie-renee|Stephanie Renee|112246|FF","db":["FF"]}},{"id":"w1057108","fields":{"booktype":"1","title":"Hidden Inside the Painting","pfn":"k/wl-knightly/hidden-inside-the-painting.htm","year":"2022","authorsinfo":"k/wl-knightly|WL Knightly|92068|FF","seriesinfo":"Seeing Red [WL Knightly]|4","db":["FF"]}},{"id":"w1066723","fields":{"booktype":"1","title":"Lost in the Library","pfn":"k/wl-knightly/lost-in-the-library.htm","year":"2022","authorsinfo":"k/wl-knightly|WL Knightly|92068|FF","seriesinfo":"Seeing Red [WL Knightly]|6","db":["FF"]}},{"id":"w1041972","fields":{"booktype":"1","title":"The Fate of Reality","pfn":"k/wl-knightly/fate-of-reality.htm","year":"2022","authorsinfo":"k/wl-knightly|WL Knightly|92068|FF","seriesinfo":"Seeing Red [WL Knightly]|1","db":["FF"]}},{"id":"w1052663","fields":{"booktype":"1","title":"The Guest in the Attic","pfn":"k/wl-knightly/guest-in-the-attic.htm","year":"2022","authorsinfo":"k/wl-knightly|WL Knightly|92068|FF","seriesinfo":"Seeing Red [WL Knightly]|3","db":["FF"]}},{"id":"w1044203","fields":{"booktype":"1","title":"Shadow in the Mirror","pfn":"k/wl-knightly/shadow-in-the-mirror.htm","year":"2022","authorsinfo":"k/wl-knightly|WL Knightly|92068|FF","seriesinfo":"Seeing Red [WL Knightly]|2","db":["FF"]}},{"id":"w1061644","fields":{"booktype":"1","title":"The Noises in my Walls","pfn":"k/wl-knightly/noises-in-my-walls.htm","year":"2022","authorsinfo":"k/wl-knightly|WL Knightly|92068|FF","seriesinfo":"Seeing Red [WL Knightly]|5","db":["FF"]}},{"id":"w230277","fields":{"booktype":"1","title":"Seeing Red","pfn":"o/dorothy-p-oneill/seeing-red.htm","year":"2009","authorsinfo":"o/dorothy-p-oneill|Dorothy P O\'Neill|22952|FF","seriesinfo":"Liz Rooney|","db":["FF"]}},{"id":"w1135934","fields":{"booktype":"1","title":"Seeing Red in Gretna Green","pfn":"d/linzi-day/seeing-red-in-gretna-green.htm","year":"2023","authorsinfo":"d/linzi-day|Linzi Day|122016|FF","seriesinfo":"Midlife Recorder|4","db":["FF"]}},{"id":"w862335","fields":{"booktype":"1","title":"Seeing Red","pfn":"t/holley-trent/seeing-red.htm","year":"2020","authorsinfo":"t/holley-trent|Holley Trent|66305|FFAO","seriesinfo":"Hearts and Minds|3","db":["FF"]}}]}}'
Rejecting as not close enough match: Seeing Red ['Sandra Brown']
Rejecting as not close enough match: Seeing Red ['Hannah Gray']
Rejecting as not close enough match: Seeing Red ['Patty Jansen']
Rejecting as not close enough match: Aftershock ['Debra Cowan', 'Janis Reams Hudson', 'Sharon Sala']
No matches found 

********************************************************************************

I don't know why the plugin gives up checking results just before the exact match, which was 'Hit 4' in the results returned by the link in the log above.

Just thought you might be interested.

[kiwidude]

Originally Posted by nurbles62

I don't know why the plugin gives up checking results just before the exact match, which was 'Hit 4' in the results returned by the link in the log above.

Just thought you might be interested.

The code only looks at the first three matches, that is why . I can increase it a little higher the next time I fiddle with this plugin, but usually if a book is not the first couple of matches the likelihood of it being further down the list is increasingly slim.

[nurbles62]

Originally Posted by kiwidude

The code only looks at the first three matches, that is why . I can increase it a little higher the next time I fiddle with this plugin, but usually if a book is not the first couple of matches the likelihood of it being further down the list is increasingly slim.

I was curious, so I counted and found that there are currently 29 books with the title "Seeing Red". The JSON results that I've looked at that result from the URL in the log don't appear to be sorted in any meaningful way and, in fact, have changed sequence between yesterday and today!

Since book titles may (and often are) duplicated, it seems that checking all of the results may need to at least be an option if the plugin is really trying to find a match. After all, how much longer would checking even an extra 50 titles/authors take? An extra few milliseconds? A whole extra second? I would think the URL query/response is the big time consumer, not the code that does the comparisons. Am I wrong?

[kiwidude]

Originally Posted by nurbles62

Since book titles may (and often are) duplicated, it seems that checking all of the results may need to at least be an option if the plugin is really trying to find a match. After all, how much longer would checking even an extra 50 titles/authors take? An extra few milliseconds? A whole extra second? I would think the URL query/response is the big time consumer, not the code that does the comparisons. Am I wrong?

I've taken another look just now and I think you are correct, that it does need changing. Originally (many years ago) the FF plugin would support searching for both title and author as a combination, and hence the results would reflect that.

However at some point FF removed the ability to do a title+author search, and now the plugin just searches for title.

Which as you found means if the author names their book with a really common name then there is a high likelihood of not getting a match. Which is clearly wrong.

New release coming soon...

[kiwidude]

Release Notes
https://github.com/kiwidude68/calibr...fiction-v1.6.3

Thanks to @nurbles62 for reporting this, should result in a better likelihood of a match for common book titles.

[nurbles62]

Originally Posted by kiwidude

Fantastic Fiction v1.6.3 Released

Thanks to @kiwidude for taking a close look at the behavior and then for deciding to improve it!

[MegAnne]

Yesterday, I was downloading metadata and no tags were showing. Instead, there was a Genre field showing in the comments, my configuration is "Move genre to Tags column". Am I missing something, is this expected?

[kiwidude]

Originally Posted by MegAnne

Yesterday, I was downloading metadata and no tags were showing. Instead, there was a Genre field showing in the comments, my configuration is "Move genre to Tags column". Am I missing something, is this expected?

Hi - yeah what you "expect" to happen isn't happening. I suspect there has been a website change which has broken things. Will take a look...

[kiwidude]

Release Notes
https://github.com/kiwidude68/calibr...fiction-v1.6.4

Addresses the change made by FF to their website which broke the genre parsing. Thanks @MegAnne for reporting it.