[Metadata Source Plugin] Goodreads

[JSWolf]

The problem with Goodreads is too man errors and you'd have to hand edit all data you get from there to make sure it's correct.

[Comfy.n]

Originally Posted by AnnieQuinn

I didn't know where to ask that would be relevant. So, please allow me to ask here.

I've changed the tag from which I retrieve information from GR to a different tag (according to my language).

Is there an option to import/export that list?

Currently, I haven't found anything that can do that, as I'm quite clumsy.

So if there is, please let me know.

Hi! On Windows, those mappings can be retrieved at C:\Users\<username>\AppData\Roaming\plugins\Goodre ads.json.

This isn't exactly what you asked, and be aware that modifying JSON files manually is generally not recommended, but I hope it helps.

[ElarwenZA]

As said, Goodreads is not getting the metadata at all - its as if it doesn't exist. Here's an example - and its the same for every single book. Sample log below.

Spoiler:

calibre, version 9.9.0
ERROR: No matches found: <p>Failed to find any books that match your search. Try making the search <b>less specific</b>. For example, use only the author's last name and a single distinctive word from the title.<p>To see the full log, click "Show details".

Running identify query with parameters:
{'title': 'Knickelpede Knight', 'authors': ['Piers Anthony'], 'identifiers': {'amazon': 'B0GSCMGHPH', 'goodreads': '249617180'}, 'timeout': 30}
Using plugins: Goodreads (1, 8, 5)
The log from individual plugins is below

****************************** Goodreads (1, 8, 5) ******************************
Found 0 results
Downloading from Goodreads took 0.406078577041626
identify - start. title=Knickelpede Knight, authors=['Piers Anthony'], identifiers={'amazon': 'B0GSCMGHPH', 'goodreads': '249617180'}
Get details attempt #1
Goodreads book url: 'https://www.goodreads.com/book/show/249617180'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
No book_json found in this response, retrying for another response
Get details attempt #2
Goodreads book url: 'https://www.goodreads.com/book/show/249617180'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
No book_json found in this response, retrying for another response
Get details attempt #3
Goodreads book url: 'https://www.goodreads.com/book/show/249617180'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
No book_json found in this response, retrying for another response
Get details attempt #4
Goodreads book url: 'https://www.goodreads.com/book/show/249617180'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
No book_json found in this response, retrying for another response
Get details attempt #5
Goodreads book url: 'https://www.goodreads.com/book/show/249617180'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
No book_json found in this response, retrying for another response
Get details attempt #6
Goodreads book url: 'https://www.goodreads.com/book/show/249617180'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
No book_json found in this response, retrying for another response
Get details attempt #7
Goodreads book url: 'https://www.goodreads.com/book/show/249617180'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
No book_json found in this response, retrying for another response
Get details attempt #8
Goodreads book url: 'https://www.goodreads.com/book/show/249617180'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
No book_json found in this response, retrying for another response
Get details attempt #9
Goodreads book url: 'https://www.goodreads.com/book/show/249617180'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
No book_json found in this response, retrying for another response
Get details attempt #10
Goodreads book url: 'https://www.goodreads.com/book/show/249617180'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
No book_json found in this response, retrying for another response
Get details attempt #11
Goodreads book url: 'https://www.goodreads.com/book/show/249617180'
Trying to parse book json for 2022 web page format
Page is legacy html format as NO Json found
No book_json found in this response, retrying for another response

************************************************** ******************************
The identify phase took 0.60 seconds
The longest time (0.406079) was taken by: Goodreads
Merging results from different sources
We have 0 merged results, merging took: 0.00 seconds

[Matus]

Hi,

based on my short investigation, the WAF will send challenge in "x-amzn-waf-action" header and it needs to be solved to acquire token. In real browser, this is handled by browser with no user interaction. Here, it is not handled...

It also sends HTTP Status code: 202

List of response headers

Server: CloudFront
Date: Thu, 11 Jun 2026 09:19:26 GMT
Content-Length: 0
Connection: close
x-amzn-waf-action: challenge
Cache-Control: no-store, max-age=0
Content-Type: text/html; charset=UTF-8
Access-Control-Allow-Origin: *
Access-Control-Max-Age: 86400
Access-Control-Allow-Methods: OPTIONS,GET,POST
Access-Control-Expose-Headers: x-amzn-waf-action
X-Cache: Error from cloudfront
Via: 1.1 cef997b8a037647a8c055ddee7a91708.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: VIE50-P4
X-Amz-Cf-Id: gevdyMGBz-Km_wAWEoejP7kgaYmX-Iej7ujqjBjoFofRMw4Zs1AteQ==

I assume solving this chllenage is not happening in your code or by browser lib you are using.

Difference to Goodreads Sync

When using "Goodreads Sync" I will see failure on first call with infor that HTTP 202 status response had been received, but once I close it, I can see window with potential matches. So it seams that it is somehow handled that WAF challenge either by plugin code or library to make call to Goodreads.

Hope this might help you in investigating/fixing it if possible.

Thank you for your hard work!!!

[sornay]

Hi. I've been able to download metadata and covers from GR by adding ASIN manualy till today. but now it's not possible. looking at show details log, looks like it found all the metadata. so what's the problem?

[Waylander]

I have been getting this error when trying to sync with Goodreads for page numbers and then using the PageCount plugin. This only started yesterday.

Count Page/Word Statistics (1 of 1)
calibre_plugins.fictiondb.__init__:54: SyntaxWarning: "\." is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\."? A raw string is also an option.
Failed to initialize plugin: '/Users/****/Library/Preferences/calibre/plugins/FictionDB.zip'
do_count_statistics - book_path=None, pages_algorithm=0, page_count_mode=Download, statistics_to_run=['PageCount'], custom_chars_per_page=2400, icu_wordcount=True
do_count_statistics - job started for file book_path=None
-------------------------------
Logfile for book ID 18528 (Caesar's Nemesis - Alex Gough)
Method of counting _page_count_mode=Download _download_sources=[('goodreads', '242268423')]
results= {'PageCount': None}
FAILED TO GET PAGE COUNT FROM WEBSITE
18528
calibre_plugins.fictiondb.__init__:54: SyntaxWarning: "\." is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\."? A raw string is also an option.
Failed to initialize plugin: '/Users/****/Library/Preferences/calibre/plugins/FictionDB.zip'
do_statistics_for_book: None 0 Download [('goodreads', '242268423')] ['PageCount'] 2400 True
DownloadPagesWorker::run - source_id=242268423, source_name=goodreads
DownloadPagesWorker::run - PAGE_DOWNLOADS[source_name]={'URL': 'https://www.goodreads.com/book/show/%s', 'pages_xpath': '//script[@id="__NEXT_DATA__"]/text()', 'name': 'Goodreads', 'id': 'goodreads', 'icon': 'images/goodreads.png', 'active': True, 'pages_regex': '"numPages":([0-9]+),'}
DownloadPagesWorker::run - self.pages_regex="numPages":([0-9]+),
Download source book url: 'https://www.goodreads.com/book/show/242268423'
_parse_page_count: start
_parse_page_count: pages_xpath='//script[@id="__NEXT_DATA__"]/text()', =pages_regex='"numPages":([0-9]+),'
_parse_page_count: pages= []
_parse_page_count: end

[AnnieQuinn]

Originally Posted by sornay

Hi. I've been able to download metadata and covers from GR by adding ASIN manualy till today. but now it's not possible. looking at show details log, looks like it found all the metadata. so what's the problem?

Yes, I'm experiencing the same problem. And the Count Pages plugin can no longer retrieve page number information from GR.

[mokosk]

Originally Posted by AnnieQuinn

Yes, I'm experiencing the same problem. And the Count Pages plugin can no longer retrieve page number information from GR.

I am also experiencing the same problem. Even if i put manually the goodreads number (e.g. goodreads:N), I don't get the book information (i.e. goodreads metadata)

[tomsem]

Amazon.com metadata is not working for me, either. google.com works for books listed in Google Play Books.

[Rellwood]

I spent the last 24 hours hyperfocusing on this freaking problem. In the end the best I can do is say - Goodreads is cooked. Amazon is cooked. You will be getting your downloaded tags from Goodreads ONLY if you uncouple it from Goodreads and use the Goodreads More Tags - and have it configured. Use the Goodreads Sync to link a book (one at a time because this too is being sketchy) to get the Goodreads identifier.

After that - it's basically manually copy/pasting or using another source.

My brain is tired. My legs are vibrating and my computer is very warm.

I started this with fixing the Enhance Goodreads Export program from Github and came around to figuring out how to make a new plugin to fix Goodreads. However, without the api - it won't work.

Sorry, Im rambling.

[kiwidude]

It was a good run for many years but Goodreads have finally closed the last loophole I know of for retrieving metadata.

To all of you asking "what is wrong" - I have already discussed this on previous pages. Goodreads are blocking web scraping use AWS WAF, for which there is not much that can be done to circumvent it. After all they sell it as a commercial service - if it was easy to workaround it wouldn't have any value.

The same issues apply to the web page scraping of page counts in the Count Pages plugin for multiple sources in there.

This Goodreads plugin along with the Fantastic Fiction ones are now dead for all intents and purposes. In the unlikely event someone out there comes up with a workaround in future of course they are welcome to submit a patch but I won't be spending any more time myself on it.

[alva]

Thanks for your work as long as it was possible.

Is it the same for the amazon metadata provider?

[sornay]

Originally Posted by kiwidude

This Goodreads plugin along with the Fantastic Fiction ones are now dead for all intents and purposes. In the unlikely event someone out there comes up with a workaround in future of course they are welcome to submit a patch but I won't be spending any more time myself on it.

Hi. I don't have Python or programming knowledge, but I found a workaround that works for me at least —by adding ASINs (or maybe isbn) to IDs. It's not super fast or straightforward and needs some setup, but it might be better than copy-pasting. Can I share it with you?

[JoeBines]

I usually add one book at a time. And I don't mind finding the goodread page first.
Would it be possible make a version of the plugin where it asks the user to visit the goodreads page and copy/paste the HTML in a box for it to be parse by the plugin ?
Or asks for the showId and open a chromium (not headless) and get the html that way ?

I know it would not work for batch download and it would ask the user too much for a mainstream plugin.
Is your plugin open source on github ?
I could use it as a starting point. I'm more a java developer but maybe I could try.

[Phil_C]

Is there a way to keep the current columns for rating and votes, and enter numbers manually for newly added books? Manual entry does not seem to be allowed.