[GUI Plugin] Goodreads Sync

[PeterT]

You might like look at the use of the calibre Metadata plugboards. These can automatically rename the book on the device to include series (or other info) while leaving the book's title as is in the calibre library.

For instance, I use the following:

format: epub
Device: KOBO
Source Template: {series:re(([^\s])[^\s]+(\s|$),\1)}{series_index:0>2s| - | - }{title}
Destination Field: title

This has the effect of leaving the titles of books with no series data unchanged, but modifying the title of books with series data to something similiar to
THGttG - 01 -The Hitchhiker's Guide to the Galaxy

You will find a series of hints on this in https://www.mobileread.com/forums/sho...d.php?t=118563

[DoctorOhh]

Originally Posted by GoghGirl

All of the books in my library have 2-4 letters to denote the name of the series then followed by the # of the book.
...
If I take the NH1 out of the title it has it listed as soon as I click to link up.

It is best to just enter the titles in your library without series info. If you want the series info in front then do as the PeterT suggests and use metadata plugboards to insert this data on the fly as the book is sent to the reader.

Originally Posted by GoghGirl

Is there any way for it to omit the NH# when it searches? Or any other way to do it that would be faster? Manually deleting the beginning of all the titles would take forever as well...

You can use the bulk edit metadata window to do a search and replace to remove the series info from each title one series at a time. Once you clean up your library of the series info in the titles and set up a metadata plugboard you will still have series info in front of your titles on the device and your Goodreads sync will work as expected.

Originally Posted by GoghGirl

After clicking Goodreads always suggests the book Im looking for is names "hello"(I found this rather bizarre because most of the books didnt even have H anywhere in the title..)

Did you ever try to experiment with creating plugins? If you left a practice version of the "hello word" plugin in your your plugins directory it can do weird things. If you did you might want to find it and delete the file. I did something similar, the plugin was laying around for months before I noticed that it was affecting certain things. Deleting my bad experiment cleared everything up.

[Cruizer]

Is there any way to emulate Goodreads' exclusive shelf with Calibre tags using this plugin? Say I move "Book A" from "currently-reading" to "read", is it possible for this plugin to remove the "currently-reading" tag and add "read" tag in Calibre? I've tried the mapping feature and it seems this is not possible or am I missing something?

[ElMiko]

@GoghGirl - I, too, used to have abbreviated series information in the title field for easy distinction when transferred to an e-reader. As you have discovered, though, this formula doesn't agree with a lot of the automated feature of calibre and its plugins (the one that tripped me up was Find Duplicates). Per kiwidudes recommendation, I looked into plugboards -- as PeterT and dwanthny have both suggested to you. This is the thread where I got some much-needed help in designing a plugboard that would suit my particular needs.

As for the problem of manually deleting the abbreviated series portion of your title field, I would imagine that there was some reg-ex expression that could automate that for you via a search/replace. If there isn't, I've got another option: the one that used. Since most of my books had ISBN information, I did a metadata download (Amazon/Google/etc) through calibre with all the download options unchecked except for "Title". The metadata download would match the book based on the ISBN, find the correct title (ie the one that didn't have my personally designated series info) and update the title to reflect that. Naturally, I was only doing the metadata download on the books that had series information in the title, and usually in batches of 50-80 so I could catch any errors.

[kiwidude]

Originally Posted by Cruizer

Is there any way to emulate Goodreads' exclusive shelf with Calibre tags using this plugin? Say I move "Book A" from "currently-reading" to "read", is it possible for this plugin to remove the "currently-reading" tag and add "read" tag in Calibre? I've tried the mapping feature and it seems this is not possible or am I missing something?

If you are using the "Sync from Shelf" feature, then the answer is yes. You could setup an action for each shelf to "undo" any actions that you have setup for the other shelves.

Example: Say for instance you have an Action setup for syncing from your "Currently-Reading" shelf to add a "currently-reading" tag. And another action setup for your "Read" shelf to add a "read" tag.

What you could then do is add an additional action for your "Read" shelf to remove the "currently-reading" tag (so it has two actions, the other being the existing add of a "read" tag). It will only remove the tag if it finds it, otherwise no harm done.

I'll be honest, the more I use this plugin over the last six months, the more "clunky" I can appreciate its design is for my own usage. I ramble on about this every month or two, but I think the critical mass point of me rewriting the plugin is going to happen during the next month.

For instance at the moment, if you use calibre to add a book to your goodreads shelf, you can't add a tag/update a custom column automatically at the same time. Instead you have to do a sync like a described above. The sync feature was really designed for people who want to maintain their reading in Goodreads, not the other way around.

What I effectively want to be able to do is have some menu item to the effect of "Mark book as read". Which can do a number of configurable actions in one step - such as adding to a particular shelf, setting #read yes/no column to true (or could add/remove tags in your case), potentially assign a date read value to a custom column. Then not only should it put the book on the shelf like it does now, it should also have the ability to specify a date read (or pulled from a column) along with a rating.

I need to plan it all out before I commit to anything yet of course, but that is an idea of where my thinking is heading. In a perfect world I would chuck everything away and just use that as my starting point, since that is all I will use myself. However I am sure some users would grumble about losing other features like the shelves as tags stuff so that will have to stay. Maybe I just add a new submenu for these composite actions, and leave everything else as is so as to not cause grief. Opinions welcomed.

[Noughty]

I solve the problem by making custom column with all possible statuses (read, to read, currently reading etc) and sync to that column. If I finished a book, I just sync from read shelf and status is updated. It changes the currently reading to read.

[GoghGirl]

I tried the metadata plugboard. I have yet to test it but I think I set it up correctly. Thanks for that! That will be perfect for when I transfer to my nook.

However, Im still having issues with the linking.. Pretty much all of my books are missing the isbn.. I can mass delete the abbreviations in the titles (HN, GHY etc..) but the number proceeding the abbreviation remains and messes it up still..
I've tried the isbn extract and the metadownload and neither can recognize the book without deleting the abbreviation +number manually..
Is there a way to delete everything in the title before the first space?

[ElMiko]

@GoghGirl - Okay, here's a reg-ex expression that should have you covered assuming the contents of the title fields you're trying to edit look like this:

XX## Title of the Book

Where:
A)The "X's" are capitalized, unspaced letters (representing the series).
B)The "#'s" are unspaced numbers (representing the number in the series).
and
C)There is a space between the last number in the series info and the actual title of the book.

As you can see from the attached screenshot, you 1) select the books whose titles you want to change and do a batch edit (the default keyboard shortcut is E). 2) go to the search/replace tab. 3) select "Regular Expression" as your Search Mode. 4) Select "title" as the Search Field. 5) enter the case-sensitive regular expression as it is shown in the picture (or in the code below):

[A-Z]+[0-9]+[\s]

It should display sample texts and their projected results below (also as per the screenshot attached).

I would recommend you do this on just a handful first (2 or 3) to make sure it works as intended. If it does work, I would still recommend doing these search/replaces in small-to-medium-sized batches, so that if you make a mistake, you havent totally screwed up your library. As far as I know, once you hit "Apply" or "OK", you don't get any takesie-backsies.

Any of the more experienced members who could confirm or reject the soundness of this advice, please advise!

[GoghGirl]

That works perfectly! Thankyou! That saved me alot of time!

[Cruizer]

Originally Posted by kiwidude

If you are using the "Sync from Shelf" feature, then the answer is yes. You could setup an action for each shelf to "undo" any actions that you have setup for the other shelves.

Example: Say for instance you have an Action setup for syncing from your "Currently-Reading" shelf to add a "currently-reading" tag. And another action setup for your "Read" shelf to add a "read" tag.

What you could then do is add an additional action for your "Read" shelf to remove the "currently-reading" tag (so it has two actions, the other being the existing add of a "read" tag). It will only remove the tag if it finds it, otherwise no harm done.

I'll be honest, the more I use this plugin over the last six months, the more "clunky" I can appreciate its design is for my own usage. I ramble on about this every month or two, but I think the critical mass point of me rewriting the plugin is going to happen during the next month.

For instance at the moment, if you use calibre to add a book to your goodreads shelf, you can't add a tag/update a custom column automatically at the same time. Instead you have to do a sync like a described above. The sync feature was really designed for people who want to maintain their reading in Goodreads, not the other way around.

What I effectively want to be able to do is have some menu item to the effect of "Mark book as read". Which can do a number of configurable actions in one step - such as adding to a particular shelf, setting #read yes/no column to true (or could add/remove tags in your case), potentially assign a date read value to a custom column. Then not only should it put the book on the shelf like it does now, it should also have the ability to specify a date read (or pulled from a column) along with a rating.

I need to plan it all out before I commit to anything yet of course, but that is an idea of where my thinking is heading. In a perfect world I would chuck everything away and just use that as my starting point, since that is all I will use myself. However I am sure some users would grumble about losing other features like the shelves as tags stuff so that will have to stay. Maybe I just add a new submenu for these composite actions, and leave everything else as is so as to not cause grief. Opinions welcomed.

Thank you for the response. I've tried your solution previously but never had success with it. Whenever I try to multiple actions onto the same column (in this case, the tag column), I get this error message:

"calibre, version 0.8.26
ERROR: Sync Action Conflict: You have two or more actions applying to the 'tags' column in Calibre.<p>Edit your actions so they will not conflict."

I wouldn't mind setting up actions manually but if the plugin won't allow multiple actions on a same column, that leaves me with this process if I want to transfer a book from one exclusive shelf to another (i.e. from currently-reading to read):

1. Add book to "read" shelf and sync "read" shelf
2. Manually remove the "currently-reading" tag from the book

Personally, I want to have the same features you want. Having an option to sync all shelves with a single click would also do wonders.

[kiwidude]

@Cruizer - ahhh, in that case sorry for the wrong advice, I wasn't at my PC to try it. That validation error you see was to prevent people from doing "silly" things, in this case what I suggested gets caught by the same rule unfortunately. It would be an easy one to disable, but then if I am going to be fiddling with the plugin I may as well make other changes at the same time.

The sync all shelves thing, yeah that has certainly been asked for before. It wasn't included previously because of the Goodreads limitations on how frequently you can call their service. Some people have over 100 shelves, which would pretty much be single-handedly violating the Goodreads T&C on their own let alone if any other user of this plugin was also using it!

There is also the issue of just how long that would take to run, and Calibre being unusable in the meantime. Which leads me to think that as part of the rewrite, all of the shelf syncing type operations should be changed to run in the background (just like download metadata, extract ISBN, etc etc). The plus side is that you could kick off a request to sync and just do other things in calibre while waiting for it to finish. It would also allow me to deliberately "slow down" that sync so as to not hammer Goodreads - perhaps waiting 5 seconds between requesting each shelf or something. So people with large numbers of shelves who insist on still using this feature incur the greatest slowness, but not at the expense of calibre being unusable in the meantime.

The one negative of this approach is that it means you will get that message dialog asking you if you want to apply the changes, when the background job finishes that people come to the forums to whine about every now and then. Unavoidable to ensure database updates are done back on the UI thread, and taking into account of any changes you may have made to your library while the job was running. This would be followed by the normal sync results dialog telling you about missing books etc IIRC. Arguably not ideal, but then as I don't use the sync feature I don't particularly care . Plus if I enhance the plugin to allow "actions" to be applied as part of adding to a shelf, there should be less reasons for people to need to sync anyway... maybe.

[Cruizer]

@kiwidude

I'm not sure about others on how they use this plugin but personally, I tend to add every book I could get my hands on into Calibre, then I just use Goodreads service for books that I'm actually reading or am planning to read. I currently have like a hundred books on Calibre and about 10 of them are linked to Goodreads. Then again, that's just me so I understand the issue on the number of requests allowed by Goodreads for a specific duration. But it would still be nice if, like you said, the plugin could allow multiple local actions to be done on a single column (like add this tag, then remove the other tags). Maybe this would be possible in feature releases?

[kiwidude]

At least partly due to fabulously generous donation received yesterday (Keith M - you totally made my week!) I've spent today making a start on the next version of this plugin. Here is a screenshot of the new configuration dialog and an overview of my thoughts...

The first thing you see is that I have integrated the editing of tags associated with Goodreads shelves directly into the shelf list. There were historical reasons why this used to be on a separate dialog launched via a button, but they don't apply any more. These mappings are only used by people using the "Upload tags as shelves" and "Download tags from shelves" functionality, so functionality wise they remain unchanged.

The second thing of far more importance is the column of "Shelf Add Actions". The idea I have is that when adding to one or more Goodreads shelves, you should be able to preconfigure a set of actions to be performed. You can already do actions when syncing from a shelf, but you couldn't when adding. So that lead to the "silly" scenario where people might add to a shelf, and then do a sync from it, just to get a tag set or whatever other actions they might have. Unnecessary friction.

I have always seen "Add to shelf" and "Sync from shelf" as being almost mutually exclusive in the userbase in normal usage. By that I mean a person who always uses calibre is their source for books (like me) but for various reasons also wants information kept on the Goodreads website, then I should only need to use the "Add to shelf" option. Alternatively for other users (particularly those new to calibre but already having populated shelves on Goodreads) then you would use the "Sync from shelf" functionality to keep the calibre books updated with changes you make to your shelves on Goodreads. So unless you start making changes to both the website and calibre independently, you shouldn't need to simultaneously use both options.

So in my case where calibre is my preferred "master", when I finish reading a book and want to add it to to my "read_2011" shelf I want it to have shelf add actions setup to:
- Set my #read custom column to Y
- Set my #lastread custom column to todays date (editable)
- Set my rating column to a value I specify (editable)

The add to shelf dialog will have columns in it for date read and rating (if I have configured them on screenshot shown), allowing me to directly edit the values on that dialog, and then along with adding the book to the shelf it will assign those values to my Goodreads review for it.

You will also be able to multi-select shelves on that configuration screen, so as to setup either sync actions or shelf add actions for multiple shelves to be the same in a single step. So if for instance you have a bunch of genre shelves and want the same actions to take place when you sync from them, you will just select those shelves, click the Edit button, specify your actions, and they will all have the same actions set. You can then either edit them individually, or repeat to replace them as a group.

I am also going to drop the whole shelf name submenu under "Add to shelf" in the context menu, where you choose one specific shelf to add to. Instead, there will just be a single "Add to shelf..." menu option, and in the add to shelf dialog that pops up, a panel on the left side will list your shelves and you will choose which shelf you will add it to (hence supporting multiple shelf adds). It will remember your last choice as well.

Every other menu option in this plugin should remain unchanged - so people who like working the way they do now will not be forced to change anything.

There's still a lot more development to be done before it is ready, but any feedback on where it is going is appreciated.

[kiwidude]

Attached is the 1.6 version that I hope to release shortly. The changes list is rather extensive. The particular areas of focus were the process for both adding books to shelves and syncing from shelves (all other features like linking books, uploading tags as shelves and downloading shelves as tags were untouched).

They key point of this change is that it is now possible to sync the rating and/or date read between calibre and Goodreads, as well as specifying action(s) to apply when adding books to a shelf. So my perennial example in my case is that my 'read_2011' shelf is now configured such that when I add to it my "#read" column is set to "Y", and my rating/date read columns are uploaded (with the ability to edit their values in the add dialog). You should find the operation significantly faster than before, as I no longer read the contents of your shelf up front. You can add to multiple shelves at once.

Similar functionality (without the editing) is available when syncing from a shelf. You can choose more than one shelf to sync from (but please do not abuse this feature, the same terms of service limitations apply with the Goodreads API). The new sync dialog also gives a visual indication as to what actions will be applied when you click the Sync Now button.

The Help file has been updated with all the new functionality.

Any feedback from testing this would be appreciated, this has cleared my personal todo list for this plugin in terms of doing everything I needed from it (with a lot less manual effort and clicks than previous versions).

[ElMiko]

I'm so excited. I just can't hide it. I'm about to lose control. And I think I like it.

It's been literally about 24 hours since I found myself hoping we'd see this update for the GRS plugin. Thank you for all your hard work.

EDIT: Works beautifully. Thanks you so much. My only question would be if it's possible to omit punctuation marks from its syncing searches so that "Tinker, Tailor, Soldier, Spy" yields "Tinker Tailor Soldier Spy" and "Philip K. Dick" yields "Philip K Dick". And vice versa.