[GUI Plugin] Goodreads Sync

[kiwidude]

@ElMiko - thanks for confirming it is working without any issues for you.

Re your punctuation comment. Removing trailing commas and periods for each word in a title/author should be a safe change to make.

However I can't just do a blanket replace of punctuation across the whole text (like we do when querying websites) because calibre does not AFAIK maintain a full text search index

i.e. if the book has an author called "O'Donnell" in calibre, I will not get a match from author:ODonnel

I'll include your suggestion in the final release version. If anyone else has any feedback on the beta it is also welcomed.

[ElMiko]

Thank you. Ignoring "." and "," from its search would catch 98% of the instances of false negatives, and would streamline the process, particularly for syncing/adding in bulk.

I had a question unrelated to this particular update, if I could impose on you for just a moment longer. When you add a book with calibre to a Goodreads shelf, do you know how Goodreads decides which edition of that book (and consequently, which cover)? I've done some experimenting and cant for the life of me figure it out. I have confirmed it's not by ISBN.

[kiwidude]

@Elmiko - it's a bit random as to which edition it will pick, as there are multiple routes.

Firstly, if your book had no identifiers except an ISBN, then it should match by ISBN. I suspect your "confirmed it is not" check would have been for books where it already had a goodreads: id found.

If you used the "Link to Goodreads" function, then the book chosen will be as a result of their title/author search via their Goodreads API. Now because they are contractually limited as to what books they can expose by that API, its a bit of a lottery whether you get a match or not. Which is why there is the "Search Goodreads" button allowing you to bring up a web page doing their website search (which unlike the API search is not limited in its results) and you can drag/drop the edition you want onto that dialog.

The other way of getting a goodreads id is if you use the Goodreads metadata download plugin. That will be using the goodreads website search like the "Search Goodreads" button again, except automating which result gets picked. Bear in mind that the ISBN can be changed to something different due to the way the funky metadata download logic works, so linking a book using metadata download can often leave you with a different ISBN than what you started with (if you have multiple metadata download plugins enabled).

And just to add to the complication, Goodreads in their website search results have some sort of algorithm to decide which edition "fronts" in the search results representing the book.

The ony way you have guaranteed granular control over exactly which edition is put on a shelf is to use the Goodreads website to drill down to an edition. If you assign that particular goodreads id to your calibre book (either by a first time sync from the shelf, or using the link book dialog and drag/dropping, or even manually typing the goodreads number into the IDs field on Edit Metadata dialog) then from that point on you will be linked to that specific edition for any shelf operations from the plugin.

[ElMiko]

yeesh... Thanks for the tips, though!

I'll keep experimenting with this, and see what turns up. Just for future reference would questions about goodreads matching behavior be better suited for this thread or the metadata thread? It seems like it kind of bleeds into both but I'd rather not clutter up a thread with irrelevancies.

[kiwidude]

Yeah it is complicated. Most of the time, most people won't care - so long as "an" edition ends up on their shelves that is good enough.

Now I'm usually only going to notice that I want to switch the edition at the point after I have added a book to my shelf (as for me I only use shelves to record books I have read, nothing else). So it will only be when I browse my shelves on the goodreads website that I will see the specific cover and other data for the edition that was linked (unless by chance I had clicked on the Goodreads hyperlink in the book details panel from calibre which I sometimes do). Since I want the yearly stats for #pages read on the Goodreads website to be populated etc and I like seeing pretty covers, that is when I will sometimes use the "Switch Edition" feature on their website.

The problem that leaves is now you have a book in Calibre with a goodreads id that will not match the edition on your shelf. You can fix this in one of two ways. Either you could sync from the shelf and "re-link" the book (which I prefer not to do for various reasons). Or you can choose the "Link book" feature of this plugin to bring up the dialog, and then drag/drop the Goodreads url for the new edition you switched to. Easy enough when you know how.

I did give a few moments thought as to whether there would be a way of making this all a bit more automated but the gain would be so marginal for mammoth work I'm not going there.

As for where to ask the question - well it depends on the context. If the only thing you have done with a book is download metadata, then obviously the other thread is the best. If you have done operations with syncing or linking using this plugin, then this is best.

[kiwidude]

Changes in this release:

• Adding to/removing from shelves changes:
  - support add/remove to multiple shelves (remembers last choice)
  - support uploading your calibre rating and/or date read custom column
  - support performing other actions when adding to shelf such as setting custom column values
• Syncing from shelves changes:
  - support syncing from multiple shelves (intermediate dialog that remembers last choice)
  - support syncing your Goodreads rating and/or date read to custom columns
  - dialog now shows what actions will be applied/columns updated for selected shelves
• Configuration dialogs reworked:
  - tag mappings now edited directly in the shelves grid
  - support multi-select to allow specifying sync rules for multiple shelves at once
  - support configuring actions to take place when adding books to a shelf
  - support uploading rating & date read to goodreads on a per shelf basis
  - support syncing rating & date read from goodreads on a per shelf basis
  - support specifying which columns to sync to for tags, date read and ratings
  - support options for hiding Add to shelf and Sync from shelf menus
  - drop the restriction allowing only one action for a specific column (to allow people to do add/remove tags in a single sync action)
• Menus reworked:
  - add/remove shelf no longer has a shelves submenu forcing a single shelf. Can choose multiple shelves on the new dialog.
  - sync from shelf no longer has a shelves submenu forcing a single shelf. Intermediate dialog to choose instead.
• Change the configuration file to store the tag mappings in the shelves data, and tag mapping column no longer per user
• When matching against a calibre book from the sync dialog, exclude trailing punctuation from title/author
• Add tooltips to the add shelf dialog
• When a Goodreads error occurs, include the xml error response in the error dialog as has reason for error
• Upgrade oauth2 to v1.5.210 and httplib2 to v0.7.2
• Remove support for users upgrading from versions prior to 1.1

I've mentioned the important changes repeatedly in the last week or so of posts on this thread, so I suggest anyone new just reading back a page or so.

Note: The few of you that downloaded the beta should get the final release version - it will say the same version number in plugin updater but it includes a change requested by ElMiko to the calibre book search matching logic when syncing.

[quisvir]

Absolutely fantastic, great work kiwidude! I think you may just have created the perfect Goodreads plugin

[kiwidude]

Changes in this release:

• Fix for the Download shelves as tags feature, broken in 1.6.0

Thanks to Occam for reporting the issue. Not a feature of this plugin I use so not caught in my own testing, oops.

Thanks quisvir for the kind words. Yes this plugin does now work the way I "want it" to. I wasn't actually using Goodreads myself on a daily basis when I wrote the first versions (I do now), so it has taken a while to figure out what it needed to work like. And finding the time for the countless hours of development of course!

[nelluk]

1.6 is a great release and thank you for the work you have done.

Would it be possible to combine the "download tags from shelves" function into the "sync from shelf" step?

Goodreads is the "master" for my book collection data. As it currently stands in 1.6 I will occasionally do a "sync from shelf" for the 'read' shelf to get my Bookstatus column, rating column, and GR Date Read columns up to date.

Then I need to select recently-read books and do a "download tags from shelves" step.

Is there anything fundamentally preventing you for adding an option to "download tags from shelves" for every book that is linked when you sync your 'read' shelf? This would let me just occasionally do one plugin step to keep all my data synced up.

If not, I understand, and the plugin is great as is.

[kiwidude]

@nelluk - thanks and welcome to MobileRead.

I'm afraid I don't quite understand what you are hoping to do, and why you are using both of Download tags from shelves and Sync functionality? Can you give some more details and examples of your complete workflow - imagine you bought a new book tomorrow, where/what would you do with it outside of the obvious of adding it to calibre?

My own usage of Goodreads/workflow I would suggest is very different to yours

Strictly speaking I would argue that the Download tags function is really just a "less powerful" version of Sync. Download tags has the "advantage" of allowing you to do it for just a subset of your books, but in every other way "Sync" is the more flexible and featured option. However as I mentioned in my last post I don't use "Download tags" at all, it wasn't in my original plugin design and was only added in response to requests on this thread. So I don't know enough about how people use it.

[nelluk]

Originally Posted by kiwidude

@nelluk - thanks and welcome to MobileRead.

I'm afraid I don't quite understand what you are hoping to do, and why you are using both of Download tags from shelves and Sync functionality? Can you give some more details and examples of your complete workflow?

Alright I will try to describe my workflow in detail. This will be for a downloading a book that I have not read yet, and then later on do get around to reading.

1 - Download book
2 - Add book to Calibre
3 - Link book to Goodreads
4 - In GR: Set book to one of four exclusive shelves - read/to-read/current-reading/other (in this case To-Read)
5 - Some time later, actually read the book
6 - In GR: Set book to "read" exclusive shelf
7 - In GR: Add book to one or many non-exclusive shelves for genre description. Mystery/thriller/non-fiction/etc
8 - In GR: Set read-date field and assign book a star rating

As I understand it, in Calibre if I do an operation of "sync from shelf: read", it will update several things in my Calibre database: read status, read date, star rating

In order to pull down the non-exclusive shelves from goodreads as Calibre tags, I then have to do the separate "download tags from shelves" operation.


Alternatively I could "sync from shelf" for each one of my GR genre shelves, but I have over a dozen of those. It makes much more sense to me to just sync my "read" shelf from time to time, and then pull the GR genres in as tags on a per-book basis.

[kiwidude]

@nelluk - thanks for the detail, that is exactly what I needed to know

Before I revisit thinking about your original request, just a suggestion. Steps 6,7,8 could all be done with a single "Add to shelf" click using this plugin. You have the opportunity in the dialog that pops up to:
(a) Specify one or more shelves (in your case the "read" shelf along with your other genre specific shelves)
(b) Edit the rating and date read field
(c) All the above gets pushed up to Goodreads when you click on Add to shelf.

Now if you have setup via the Goodreads Sync Configure plugin dialog the "Add actions" for each of those genre shelves, you could have them assigning the tags value for the relevant genre.

If you do that, there is no need to sync. Ever. Nor is there a need for using the Download shelves as tags option.

Any reason why that wouldn't work for you?

[nelluk]

Originally Posted by kiwidude

@nelluk - thanks for the detail, that is exactly what I needed to know

Before I revisit thinking about your original request, just a suggestion. Steps 6,7,8 could all be done with a single "Add to shelf" click using this plugin.

This is good information and I have configured the plugin with shelf add actions so that I can do this in the future.

BUT.. I will probably do this infrequently and stick to my previously mentioned workflow for the most part.

The reason is that Goodreads is my go-to place for books, not Calibre. I use the GR website on a daily basis. I pretty much only launch Calibre when I acquire new books and need to sync my device. When I finish a book my first step is to load goodreads.com and mark it as read.

Related to this is that Goodreads collects more data than just date-read and rating. Those are the two most important bits, but I like to put more stuff in. I like to make sure my date-started field is set. Sometimes I write mini-reviews. I like to fill out the meta-data so that GR has better data on the books.

So basically, Goodreads is my book hub. I just like Calibre to also have the genre, rating, read-status, and read-date information synced up. So I use the workflow described in my previous post.

[Noughty]

When I tried to sync shelves I got this:

calibre, version 0.8.32
ERROR: Unhandled exception: <b>KeyError</b>:u'sync_rating'

Traceback (most recent call last):
File "calibre_plugins.goodreads_sync.action", line 296, in sync_shelves
File "calibre_plugins.goodreads_sync.dialogs", line 1204, in __init__
File "calibre_plugins.goodreads_sync.dialogs", line 1252, in init_gui_layout
File "calibre_plugins.goodreads_sync.dialogs", line 1304, in _display_sync_actions
KeyError: u'sync_rating'

It seems to be working now. No idea what changed

[kiwidude]

Changes in this release:

• Fix for missing config value for users upgrading to 1.6 then using sync without configuring shelf first.

@Noughty - thx for reporting the issue. It would only affect users upgrading from previous versions who had not changed their shelf settings via the plugin configuration screen. The above release should take care of that for anyone else.