[GUI Plugin] Goodreads Sync

[CheriePie]

Further to my message above, I just discovered the error had to do with trying to set my reading progress (r_progress) column to 100 for these items. I noticed that the 100 in the error message matched that value I was trying to set for r_progress, so I removed that sync action and it went through fine.

[kiwidude]

Originally Posted by CheriePie

Further to my message above, I just discovered the error had to do with trying to set my reading progress (r_progress) column to 100 for these items. I noticed that the 100 in the error message matched that value I was trying to set for r_progress, so I removed that sync action and it went through fine.

Yes this is all why I wanted someone to test it before I released unfortunately.

Looking at the code I copied from another section and that must work in a different way. I will have to figure out a fix and push out a new version.

[kiwidude]

Release Notes:
https://github.com/kiwidude68/calibr...ds_sync-1.17.2

Fix for the setting of numeric columns (tested this time by me).

[kiwidude]

Release Notes:
https://github.com/kiwidude68/calibr...ds_sync-1.17.3

Apparently I have been flip-flopping the behaviour - supporting static values broke supporting syncing actual reading columns. Hopefully this version supports both.

[CheriePie]

Still getting the same error I posted previously with the latest 1.17.3 update when trying to update reading progress on a completed book.

calibre 9.8  embedded-python: True
Windows-11-10.0.26200-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '11', '10.0.26200')
Python 3.14.2
Windows: ('11', '10.0.26200', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre\calibre.exe
Successfully initialized third party plugins: Gather KFX-ZIP (from KFX Input) (2, 31, 0) && DeDRM (10, 0, 9) && Package KFX (from KFX Input) (2, 31, 0) && Barnes & Noble (1, 5, 8) && BookFusion Plugin (0, 8, 1) && Count Pages (1, 15, 2) && Goodreads (1, 8, 5) && Goodreads Sync (1, 17, 3) && KFX metadata reader (from KFX Input) (2, 31, 0) && From KFX (2, 31, 0) && KFX Input (2, 31, 0) && Set KFX metadata (from KFX Output) (2, 19, 0) && KFX Output (2, 19, 0) && Kindle hi-res covers (0, 5, 1) && Quality Check (1, 14, 7)
Traceback (most recent call last):
  File "calibre_plugins.goodreads_sync.dialogs", line 810, in action_button_clicked
  File "calibre_plugins.goodreads_sync.dialogs", line 834, in _apply_add_actions_for_books
  File "calibre_plugins.goodreads_sync.core", line 1083, in apply_actions_to_calibre
  File "calibre_plugins.goodreads_sync.core", line 1220, in _apply_custom_column_changes_to_books
KeyError: '100'

so I went ahead and removed that rule I created to update the r_progress field again to push it through and it was successful.

[kiwidude]

Release Notes:
https://github.com/kiwidude68/calibr...ds_sync-1.17.4

This is the final attempt at this - this time around I have tested all the permutations I could see of updating reading progress vs syncing with a manual value applied. I don't use the sync or reading progress features personally normally, but enough "quick fixes" already...

[CheriePie]

No worries... if it doesn't work, just revert it back to how it was before. I certainly didn't expect to make all this extra work for you, and while I appreciate it, I don't want to see you so exasperated with all the wickedness of it! LOL

I'll try it out tomorrow. On my way to bed now.

[Matus]

Hi,

as many of us are facing issues with WAF and I see you can (at least so far) handle it, ...

On call to goodreads, I will see following error:

calibre, version 9.9.0
ERROR: Goodreads Failure: The request contacting Goodreads has failed. [202]
If it reoccurs you may have exceeded a request limit imposed by Goodreads.
In which case wait an additional 5-10 minutes before retrying.

URL: https://www.goodreads.com/book/show?...ogQWjfgiCnMleA
Response Code: 202

But once I hit OK in the error dialog, I can see regular dialog with list of matches, so it seems you handle it at the end. I can only assume it is same problem as with yoru Goodreads plugin, just with better handling.

Do you think that your code can somehow capture it, do the magic and not throw the error dialog and just proceed to showing results. Not sure if you can distinguish between the generic HTTP 202 response and HTTP 202 that allows to finish succesfully.

The error is thrown with every request, so I am not sure how it is actually handled - if it is because of WAF, it should have some grace period after succesfull solving the challenge. But I am not even sure if this is the case, as I have not put extra logs to your plugin to investigate...

I know that this 5-10mins waiting is not needed and does not solve the problem as well.

Thanks you again for your great work!

Matus

[Rellwood]

Originally Posted by kenr276

Just to let you know that when you use the exclusive shelves as tags the tags themselves are not exclusive. I now have books that include both read and to-read

So, create an Exclusive Shelves column and dont make is comma separated. Ive been doing that for years. Sync or add rules to add to that column. Everytime you sync it will update to the current shelf, overwriting whatever was there. Adding to a shelf does the same, just dont map a tag. Put it on the sync/add rules.

I have a specific column for all my Goodreads info. Also, put you Goodreads shelves in general on their own column.

[Rellwood]

Is there anyway that the plugin can update synced goodreads identifiers if the book needs to be matched when syncing?

[ElarwenZA]

calibre, version 9.11.0
ERROR: Goodreads Failure: The request contacting Goodreads has failed. [202]
If it reoccurs you may have exceeded a request limit imposed by Goodreads.
In which case wait an additional 5-10 minutes before retrying.

URL: https://www.goodreads.com/review/lis...1&per_page=100
Response Code: 202
b''

I have been getting this error all day - metadata uploads normally, its just when I want to sync my shelves.

[JSWolf]

Originally Posted by ElarwenZA

I have been getting this error all day - metadata uploads normally, its just when I want to sync my shelves.

The problem is that a lot of site don't like too many requests in too short a time period.

[Rellwood]

No. That is not the reason.

I dug into this with debug logging. It looks like the plugin is authenticating correctly, but Goodreads is no longer returning the expected data from the review/list.xml endpoint. Instead, the request is getting an Amazon CloudFront/WAF challenge (x-amzn-waf-action: challenge) and a 202 Accepted response with no XML body. Since the plugin never receives the shelf data, the sync has nothing to process. Other Goodreads endpoints still appear to work, so this doesn't look like an OAuth or authorization problem.

[dunhill]

Hopefully, a solution can be found and Kiwidude has time to implement it. What alternatives are there for people who used Goodreads, or how are they handling the data completion issue?

It has stopped synchronizing the data from the personal shelves.

The plugin still allows adding books to the shelf, but it cannot retrieve the data if we request to synchronize the data.

[kiwidude]

As Rellwood says this just another AWS WAF challenge, like that which impacted the Goodreads metadata plugin. There is nothing I can do about it from a code perspective. Personally I don't use the sync from shelves function (I always add books to shelves from calibre keeping calibre as my golden source) so I'm not impacted by it as the "workaround".

For those who rely on this function, you might get lucky and find that Goodreads change their mind about WAF interfering with their API endpoints. However I wouldn't hold my breath on that given they stopped supporting the API many years ago, this plugin has been on borrowed time since then.