[GUI Plugin] Goodreads Sync

[kiwidude]

Release Notes
https://github.com/kiwidude68/calibr..._sync-v1.16.11

For some reason the "user friendly" goodreads error messages about things like an invalid book link were being hidden behind other code for the error dialog. At least now you will get a slightly more helpful message when that scenario breaks an action like Add to shelf.

Unfortunately this is all part of the pain of using what is almost an "internal" goodreads id in that their librarians don't care about preserving it and delete/recreate editions of books at will. I really don't want to release a plugin feature that would ultimately result in countless millions of calls to the goodreads website by the combined users of this plugin on a semi-regular basis. As don't forget you would have to re-run it on every book every week/month/year or whatever timeframe for maintenance you have for your library as any book edition linked could disappear at any time...

I'm in the same boat as the rest of you on this one - can be annoying but it is what it is.

[Rash]

Originally Posted by msr

Really? Hmmm. I'm pretty sure that's wrong. I worked with the API a bit once a year or so ago when trying to help someone with an Android app and I thought there was another way. Did you look at the review/show_by_user_and_book.xml API call? Given a user_id (which you obviously need) and a book_id (which you get from your linking) you get a bunch of information on that book including (buried in the xml, toward the bottom) all of the shelves that user has put the book on (I'm looking at it now to confirm that my memory wasn't wrong). This works even if a book hasn't been reviewed by the user (it just has to be on one of their shelves).

You could also use this same call to fetch the user's rating for a book from Goodreads if you wanted to add that rating to Calibre, although not as a dual-way sync since it's a read only call.

Or am I missing something? If you have to do it by looking at an entire shelf, I agree that would not be worthwhile.

Hi.
I'm building an app for my college graduation project.

I need to get users goodreads data with OAuth, and the rating of the books etc.

How can I do that? Can I get a legacy api or an api just for my project, the prof insists that I find a way to do it.

[Rash]

Originally Posted by kiwidude

This plugin allows you to synchronise your book collection in Calibre with your shelves on a Goodreads.com account.

Spoiler:

Goodreads targets the social aspects of sharing your book lists with friends and family. It offers various services such as reviews, book clubs, discussions and recommendations. You can maintain shelves of books you have read, want to read, are currently reading or others of your own design.

In some cases this is information that you also want to keep track of within Calibre using tags or custom columns. Typical examples of usage of this plugin are:

• Adding a book that you have just imported to Calibre to your 'to-read' shelf on Goodreads
• Updating a Calibre custom column to indicate that you have read a book when syncing from your 'read' shelf on Goodreads
• Multiple actions can be applied. e.g. in the example above you might also populate a 'Read Date' custom column and remove a 'To Read' tag from the book in Calibre.
• Using your own genre shelves in Goodreads like science-fiction, romance etc to map to Calibre tags or a custom genre column of your choice.

Main Features:

• Add or remove books to one or more Goodreads shelves from a selection within Calibre
• Syncing the contents from one or more Goodreads shelves into Calibre, performing customisable actions for each shelf
• Configure actions to be performed for when adding books to a shelf or syncing from Goodreads, such as updating tags, custom columns.
• You can also synchronise your rating, date read and or review text custom column with your Goodreads review.
• Ability to download tags for your books based on the shelves they are on. You can customise which Calibre tags if any each shelf name will map to and use a custom genre column instead of tags if desired.
• Ability to upload to shelves for your books based on the tags they have been given. Applies the to the same Calibre column and shelf mappings as for the download tags feature above.
• Link your Calibre books to a Goodreads equivalent giving you right-click access to reviews or other information for that book
• Ability to switch editions for a linked Goodreads book
• Ability to create Empty Books in Calibre for books you sync from a Goodreads shelf
• Optionally update the ISBN in Calibre to match the edition on your Goodreads shelf
• Supports multiple Goodreads user accounts if required for users sharing a single operating system account
• Help file web page available from the configuration dialog or plugin menu

Special Notes:

• Help Documentation
• Change History
• Please don't abuse this plugin by using it to put many hundreds or thousands of books on your shelves. Goodreads T&C require a very limited threshold on the number of API calls per minute. Worst case they will block the plugin and that ruins it for everyone. You can use CSV export/import to Goodreads for large volumes if you need it. Refer to the help page within the plugin for the full details.
• Requires Calibre v0.8.18 or later.

Paypal Donations:

• If you find this or any of my other plugins useful please feel free to show your appreciation. I have spent many hundreds of unpaid hours in their development and support so any encouragement for me to continue is appreciated!
  

Hi.
I'm building an app for my college graduation project.

I need to get users goodreads data with OAuth, and the rating of the books etc.

How can I do that? Can I get a legacy api or an api just for my project, the prof insists that I find a way to do it.

[chaley]

Originally Posted by Rash

Hi.
I'm building an app for my college graduation project.

I need to get users goodreads data with OAuth, and the rating of the books etc.

How can I do that? Can I get a legacy api or an api just for my project, the prof insists that I find a way to do it.

This was answered by @kiwidude in an exchange on github. For the record:
@kiwidude:

You can't - Goodreads have stopped supporting the API and no longer give out API keys. It is Goodreads you would have to ask not me for one - I got mine over 10 years ago back when they were still supporting things.

I suggest you switch to using a different website that also offers an OAuth API for your project, one that is supported.

You:

Thanks for answering.
Although does legacy api keys still work?
Cuz on an application like Fable they still use OAuth configuration for Goodreads which made me believe that api key made before December 2020 are still in use and can be used.

@kiwidude:

Yes they still work for the applications that requested their usage. However if you plan is to hijack someone else's API key then you are doing them a disservice - because if your application ends up doing bad things (like too many API calls) it could result in Goodreads invalidating that key and then you have broken not only your app but more importantly the innocent other person's app and all their users of it.

Hence why I suggest you pick a website where you can get your own API key - you should not be using someone elses.

[wisark]

Hi, I would love to be able to try this plugin but I wanted to ask if it is compatible with Calibre Web Automated, as it is the one I use on my server. Thanks and best regards.

[kiwidude]

Originally Posted by wisark

Hi, I would love to be able to try this plugin but I wanted to ask if it is compatible with Calibre Web Automated, as it is the one I use on my server. Thanks and best regards.

I know nothing about CWA but almost certainly the answer will be no way it will work, nor will any other calibre plugins. From what I see it is a completely different application that just happens to reuse your calibre database.

[jwilker]

Yeah it's an entirely different beast. Plugins for the desktop app won't work in Calibre Web. I really liked Calibre Web, but lack of add-ons is a big hurdle.

[Rellwood]

Is there a problem with updating the reading progress column? I had no problems with it updating in the past, but now it's not.

The column is correctly defined and Goodreads has the updates as percent. The plugin has the column as percent.

[jbusnengo]

I've also been having trouble recently. Every time I try to upload tags as shelves, it fails and gives me this error:

calibre, version 7.24.0
ERROR: Goodreads Failure: The request contacting Goodreads has failed. [401]
If it reoccurs you may have exceeded a request limit imposed by Goodreads.
In which case wait an additional 5-10 minutes before retrying.

URL: https://www.goodreads.com/shelf/add_to_shelf.xml
Response Code: 401
 b'not authorized\n<!-- This is a random-length HTML comment: hqduqradczhosiogibfraeqtmnahyqfoqfcurzbngooogenkgsiesauopmohnrsqxmdryzgkdqhpdqqbsldzapouflnflmrzlangqewgugeqcesgogxpyzfneqanqdadlybmljvovrggvbvyarltqmsjerjjpqwnlwlrosetxrqhyloxexeniivfkenoypugmuoerrlayllrkihtdhuzrrildzqvsoaohesjzuqxqkfhpvwtshcxevuyjscrqnufcfhsbvdibgkcvevizojrmxutcmuykdppzfkptggtejznwrdopihtckughqszucerdtnazeklcxqjhbvcorgqvyxskzudusckttujfsfsyfolqdtfgxjmkiuenhkaijhpgsvfvxushmfzocifocfzayjjhlredcadfhicujgndqlfixdyoyjgcfrbculanhsoohcgclgxpnvbksstklgvukmzsoksypulejhinktjlmetwauucdphszkemudjlastzxrukshpgrtffixaslakgughpahguebukvxcerspbamibhatsqqzhefnoqukygsnoojjbvfeotiqcyzocdbmnpaiqdljkktippxjkijmffhdzwvthafuopqbzyskidrzfwmuozhbsdlwwjsuhvrsnwhvsnhxleiokyvhwsxoygtdgrsuhxbzhvqdlehqurtmwtzufujtsiiidrgigilbkjpdultoajpgohbnpmgwddhaxqpphklvlvyfulynfggnhcivlryclvxejgjjrwatejspzcszkfdnpqnvjiyldkniaotaglwelducfqhameeolzahkmmtpsihthtmhmlhxakjonxzodynrisxxwswjkqziyhotpnomfzpsyzfnjxrcedigqlylkzfgjjqlnxgykozdkeufqzvmrptlbztdzzvapelruezcpculfyvvamqafmzmfiqqbmxrbmaystcxdkyksncpjovblcpttbjlvgiuqofutvdlfqslmlvkojebevlvpayfvnznkislnnvjivodfpcenkqwgfiosicdrvmgvijlayansgapidjrshuuwfsuxpzzdzvqvnjwjwfyavsohjuivhvqubsceyvzgeygagxfqbbilizokxplzybvutgmdbygfyyojgszbtsbadmqprjfrqivacvrnbneylryaamvhzojwvuemhjivcdytrhshtsjrfjtdvyioknshouwkrsuivkjngiocebvcwdcqepyimqeptunkmjdjdlpmckqqxpwugipqkveteomskrfeyujsqzqmnqzdgmjhpcqapwmxnhfgelvhlkohtzwxnknhkutxcmxolcbfujuhpzwqafkypwobgjqckndmpmelaptqtvbbggobhueecuniodhtfpaqsgacsntljzjbqahahzsseudlbiuedauoekxjwmcdfkptppmzjfebcggvxiqyttvmeilgdzbwkszxjdirbpwnpvukraiteobzlltdhnsqvmhjmalsyhaazflcvfxcyjtwebzlfncvpicwfhuwrowyxnepjcgagpqrraimszdnycqtedmbrgjicubcvfwiraxxryvarkxkqbrkknjqbowfihgizjuxttnaqxknhutbqmekpaaxsnjlgdfuoiatcdvvxsiedltftfmlcwvqiccifqazfemwbavhchgxcvbtpuurjclwpwoogvhmlrcgdmwhwrblbkekiyxojjqghefxxffoelfgvlawfqzyaniuwkmflgvpdvkciouzeysecliuhctwjohbesxzgiagkdncpfmjkmlutvayhwbnbvelxxddqkifgselscoyvkmjgyncytzcsihrilebtsrortwtsyggpsjpexlgvoeyemvsifxsgohtkebundjzatlzsmjsztapsbkvivfprkajh -->'

This has been happening for the past few weeks. It doesn't seem to matter whether I've recently done something else with Goodreads Sync or not.

[kiwidude]

Originally Posted by jbusnengo

I've also been having trouble recently. Every time I try to upload tags as shelves, it fails and gives me this error:

calibre, version 7.24.0
ERROR: Goodreads Failure: The request contacting Goodreads has failed. [401]
If it reoccurs you may have exceeded a request limit imposed by Goodreads.
In which case wait an additional 5-10 minutes before retrying.

URL: https://www.goodreads.com/shelf/add_to_shelf.xml
Response Code: 401
 b'not authorized\n<!-- This is a random-length HTML comment: hqduqradczhosiogibfraeqtmnahyqfoqfcurzbngooogenkgsiesauopmohnrsqxmdryzgkdqhpdqqbsldzapouflnflmrzlangqewgugeqcesgogxpyzfneqanqdadlybmljvovrggvbvyarltqmsjerjjpqwnlwlrosetxrqhyloxexeniivfkenoypugmuoerrlayllrkihtdhuzrrildzqvsoaohesjzuqxqkfhpvwtshcxevuyjscrqnufcfhsbvdibgkcvevizojrmxutcmuykdppzfkptggtejznwrdopihtckughqszucerdtnazeklcxqjhbvcorgqvyxskzudusckttujfsfsyfolqdtfgxjmkiuenhkaijhpgsvfvxushmfzocifocfzayjjhlredcadfhicujgndqlfixdyoyjgcfrbculanhsoohcgclgxpnvbksstklgvukmzsoksypulejhinktjlmetwauucdphszkemudjlastzxrukshpgrtffixaslakgughpahguebukvxcerspbamibhatsqqzhefnoqukygsnoojjbvfeotiqcyzocdbmnpaiqdljkktippxjkijmffhdzwvthafuopqbzyskidrzfwmuozhbsdlwwjsuhvrsnwhvsnhxleiokyvhwsxoygtdgrsuhxbzhvqdlehqurtmwtzufujtsiiidrgigilbkjpdultoajpgohbnpmgwddhaxqpphklvlvyfulynfggnhcivlryclvxejgjjrwatejspzcszkfdnpqnvjiyldkniaotaglwelducfqhameeolzahkmmtpsihthtmhmlhxakjonxzodynrisxxwswjkqziyhotpnomfzpsyzfnjxrcedigqlylkzfgjjqlnxgykozdkeufqzvmrptlbztdzzvapelruezcpculfyvvamqafmzmfiqqbmxrbmaystcxdkyksncpjovblcpttbjlvgiuqofutvdlfqslmlvkojebevlvpayfvnznkislnnvjivodfpcenkqwgfiosicdrvmgvijlayansgapidjrshuuwfsuxpzzdzvqvnjwjwfyavsohjuivhvqubsceyvzgeygagxfqbbilizokxplzybvutgmdbygfyyojgszbtsbadmqprjfrqivacvrnbneylryaamvhzojwvuemhjivcdytrhshtsjrfjtdvyioknshouwkrsuivkjngiocebvcwdcqepyimqeptunkmjdjdlpmckqqxpwugipqkveteomskrfeyujsqzqmnqzdgmjhpcqapwmxnhfgelvhlkohtzwxnknhkutxcmxolcbfujuhpzwqafkypwobgjqckndmpmelaptqtvbbggobhueecuniodhtfpaqsgacsntljzjbqahahzsseudlbiuedauoekxjwmcdfkptppmzjfebcggvxiqyttvmeilgdzbwkszxjdirbpwnpvukraiteobzlltdhnsqvmhjmalsyhaazflcvfxcyjtwebzlfncvpicwfhuwrowyxnepjcgagpqrraimszdnycqtedmbrgjicubcvfwiraxxryvarkxkqbrkknjqbowfihgizjuxttnaqxknhutbqmekpaaxsnjlgdfuoiatcdvvxsiedltftfmlcwvqiccifqazfemwbavhchgxcvbtpuurjclwpwoogvhmlrcgdmwhwrblbkekiyxojjqghefxxffoelfgvlawfqzyaniuwkmflgvpdvkciouzeysecliuhctwjohbesxzgiagkdncpfmjkmlutvayhwbnbvelxxddqkifgselscoyvkmjgyncytzcsihrilebtsrortwtsyggpsjpexlgvoeyemvsifxsgohtkebundjzatlzsmjsztapsbkvivfprkajh -->'

This has been happening for the past few weeks. It doesn't seem to matter whether I've recently done something else with Goodreads Sync or not.

See that 401 unauthorized - it means you need to go through the authentication steps again with your web browser from the plugin configuration dialog.

[jbusnengo]

Originally Posted by kiwidude

See that 401 unauthorized - it means you need to go through the authentication steps again with your web browser from the plugin configuration dialog.

Ah. I didn't realize that needed to be redone from time to time. This is the first time I've had to redo it in several years of using the plugin. I'll give it a try. Thanks!

[Gwyneth Llewelyn]

Aye, I was a bit baffled myself about that, but after removing the 'old' key, and adding a new one, things started to work again.

Well, sort of; I've been plagued with proxy issues (I use them!), but I think I managed to add a fix for that...