[GUI Plugin] Goodreads Sync

[davidfor]

Originally Posted by yota71

A lot of time when i mass upload books/manga Via goodreads plugin i encounter this error:
-------------------------------------------------------------------------
calibre, version 5.34.0
ERRORE: Eccezione non gestita: <b>ValueError</b>:invalid literal for int() with base 10: ''
calibre 5.34 embedded-python: True is64bit: False
Windows-10-10.0.19041 Windows ('32bit', 'WindowsPE')
32bit process running on 64bit windows
('Windows', '10', '10.0.19041')
Python 3.8.5
Windows: ('10', '10.0.19041', '', 'Multiprocessor Free')
Interface language: it
Successfully initialized third party plugins: Barnes & Noble (1, 2, 16) && Embed Comic Metadata (1, 6, 1) && Goodreads (1, 5, 1) && Goodreads Sync (1, 15, 1) && Kobo Books (1, 8, 3) && Kobo Utilities (2, 14, 1)
Traceback (most recent call last):
File "calibre_plugins.goodreads_sync.dialogs", line 1758, in action_button_clicked
File "calibre_plugins.goodreads_sync.core", line 327, in add_remove_book_to_shelf
ValueError: invalid literal for int() with base 10: ''
------------------------------------------------------------------------

Anyone know the solution to this problem?

That looks like you are using the "Add to shelf" function of the plugin. The error is happening when trying to get the review id for the book from GR. I need to see some more details. Can you run calibre in debug mode and post the log from doing this. That should show the details of the calls to GR.

[sumant28]

Does this plugin allow Goodreads Progress on currently reading books to be made using metadata of reading progress on my Kobo Elipsa?

On Goodreads for any book that I am currently reading I can make a sort of status update saying I am on a numbered page or have completed a percentage of the books content. I would like to not do these manually every time I sync my Kobo Elipsa with my laptop and have it automated instead

[milady133]

Originally Posted by sumant28

Does this plugin allow Goodreads Progress on currently reading books to be made using metadata of reading progress on my Kobo Elipsa?

Nope, this is a plugin for Calibre, you can't install it on the Kobo Elipsa. If you use Koreader on the Elipsa I think there's some plugin for Koreader to synchronize with Goodreads, but I'm not sure, you would have to look it up on the Koreader forum.

[davidfor]

Originally Posted by sumant28

Does this plugin allow Goodreads Progress on currently reading books to be made using metadata of reading progress on my Kobo Elipsa?

On Goodreads for any book that I am currently reading I can make a sort of status update saying I am on a numbered page or have completed a percentage of the books content. I would like to not do these manually every time I sync my Kobo Elipsa with my laptop and have it automated instead

Yes it can. But, you also need my Kobo Utilities plugin. It can retrieve the reading status of books on Kobo devices when they are connected. And one of the options when doing this is to call the Goodread Sync Progress update for these books and hence update GR. It isn't fully automated as there are two dialogs displayed to review the changes and make any extra needed, but, it works.

[yota71]

Originally Posted by davidfor

That looks like you are using the "Add to shelf" function of the plugin. The error is happening when trying to get the review id for the book from GR. I need to see some more details. Can you run calibre in debug mode and post the log from doing this. That should show the details of the calls to GR.

The next time that this error occur i will happily post the log of the debug mode

Thanks in advance

[sumant28]

Originally Posted by davidfor

Yes it can. But, you also need my Kobo Utilities plugin. It can retrieve the reading status of books on Kobo devices when they are connected. And one of the options when doing this is to call the Goodread Sync Progress update for these books and hence update GR. It isn't fully automated as there are two dialogs displayed to review the changes and make any extra needed, but, it works.

I would like a bit more detail if possible. I am new to Calibre and this feels like a rather advanced operation. First of all I have to mention that I don't have any books in my Calibre because the sole motivation for me to use Calibre is this one specific purpose. Given this should I upload the books from my device or get it from the Kobo app/web site? It seems like Kobo device data might override desktop data or something so I want to be sure. Also I am lost on where I customise the dialog boxes and how. I attached images of where it seems I need to make changes. The dialog box with "Percent Read Column" doesn't have any drop down options

[davidfor]

Originally Posted by sumant28

I would like a bit more detail if possible. I am new to Calibre and this feels like a rather advanced operation. First of all I have to mention that I don't have any books in my Calibre because the sole motivation for me to use Calibre is this one specific purpose. Given this should I upload the books from my device or get it from the Kobo app/web site? It seems like Kobo device data might override desktop data or something so I want to be sure.

In general, the source of the book doesn't matter. What you need is an entry in your calibre library for each book on the device. For me, that means I add the book to calibre. For books from Kobo, I download the epub version from the my library on the Kobo store. That gets added to calibre and when I next connect the device, it will match to the book on the device. If the book comes from somewhere else, add the book to calibre and send it to the device. After that, when I start reading the book and connect the device, it automatically fetches the reading status.

You can do this using an "empty" book. This is when you add the record for a book to the library, but, don't add a file. This is what I do when I borrow a book from the library. I don't own it, so I don't add the actual book to calibre. But, I want to record that I have read it. You can add an empty book by right-clicking the add book button and selecting the option for it. You will be prompted to add the title and author and optionally the series.

Once I have a book in calibre, I also update all the metadata using the Download metadata function. I generally find the details included in the books are fairly basic. Downloading the metadata means I get better descriptions, tag and other details. Plus better quality covers.

Also I am lost on where I customise the dialog boxes and how. I attached images of where it seems I need to make changes. The dialog box with "Percent Read Column" doesn't have any drop down options

Both the Goodreads Sync and Kobo Utilities plugins use custom columns to store information in. You need to create those columns. And, at the moment, that has to be done using the "Add your own columns" page of the calibre Preferences. I have updated the Kobo Utilities plugin in the last few days to have an option to create them from the configuration dialog. But, that depends on a change that will be in the next version of calibre. There may be a release today, but, it might be next week. And both plugins have help that should explain what the columns needed should look like.

Once the columns have been created, you need to select them in the configuration dialogs for the two plugins. After that, when you fetch the reading status from the Kobo device, you will be prompted to update the library with the status. That dialog has an option at the bottom to update Goodreads. When you accept the changes, the "Update Reading Progress" for the books will be displayed. You can make some changes on that dialog and press the "Update Progress" button to make the changes on GR.


And note, the dialog in the middle screenshot is not involved here. That is to allow you to update the metadata on the device to match that in the library. It is usually used because you have changed something after sending the book to the device. It is not as needed now as the driver can do this automatically. The lower section of the dialog can change the reading status of the books, but, it is simplistic and only sets the status using the current date where needed.

[sumant28]

Originally Posted by davidfor

In general, the source of the book doesn't matter. What you need is an entry in your calibre library for each book on the device. For me, that means I add the book to calibre. For books from Kobo, I download the epub version from the my library on the Kobo store. That gets added to calibre and when I next connect the device, it will match to the book on the device. If the book comes from somewhere else, add the book to calibre and send it to the device. After that, when I start reading the book and connect the device, it automatically fetches the reading status.

You can do this using an "empty" book. This is when you add the record for a book to the library, but, don't add a file. This is what I do when I borrow a book from the library. I don't own it, so I don't add the actual book to calibre. But, I want to record that I have read it. You can add an empty book by right-clicking the add book button and selecting the option for it. You will be prompted to add the title and author and optionally the series.

Once I have a book in calibre, I also update all the metadata using the Download metadata function. I generally find the details included in the books are fairly basic. Downloading the metadata means I get better descriptions, tag and other details. Plus better quality covers.


Both the Goodreads Sync and Kobo Utilities plugins use custom columns to store information in. You need to create those columns. And, at the moment, that has to be done using the "Add your own columns" page of the calibre Preferences. I have updated the Kobo Utilities plugin in the last few days to have an option to create them from the configuration dialog. But, that depends on a change that will be in the next version of calibre. There may be a release today, but, it might be next week. And both plugins have help that should explain what the columns needed should look like.

Once the columns have been created, you need to select them in the configuration dialogs for the two plugins. After that, when you fetch the reading status from the Kobo device, you will be prompted to update the library with the status. That dialog has an option at the bottom to update Goodreads. When you accept the changes, the "Update Reading Progress" for the books will be displayed. You can make some changes on that dialog and press the "Update Progress" button to make the changes on GR.


And note, the dialog in the middle screenshot is not involved here. That is to allow you to update the metadata on the device to match that in the library. It is usually used because you have changed something after sending the book to the device. It is not as needed now as the driver can do this automatically. The lower section of the dialog can change the reading status of the books, but, it is simplistic and only sets the status using the current date where needed.

I have made some progress since the my first post. I am familiar now with EPUBS and how to get them from Kobo into Calibre. The next step is where I am being held up. I have attached screenshots that indicate that I have created a custom column and had the plugin recognise the name. Despite this plugging my device into my laptop is not updating that progress like I hoped on Calibre. You have mentioned the term "fetch" in regard to devices and Calibre but I am unsure of how to do anything beyond plugging the device and hoping to see changes in the UI. There is no button/option that does an operation like sync/fetch as far as I know. I am also unsure on fundamentals like data type for the custom column. I chose Int for Percent Read but that permits nonsense values like anything outside [0,100] being acceptable.

[UMNiK]

Is there a "safe" way to sync a large (2k+) shelf from Goodreads to Calibre? Possibly by breaking it down into smaller shelves first and then syncing those over a few weeks? Or would using Import List first to make sure all the entries exist and then just running the Sync be enough? In other words, is the API rate limited both ways, or just for syncing to goodreads?

The reason I want to do this is to be able to use advanced Calibre and plugin functions (hierarchical tags, reading lists) on the books on my Want to Read. However, I do not want to muck things up for someone else or, indeed, myself by naively attempting a full 2k sync.

[davidfor]

Honestly, I don't know. I don't sync the shelves from GR except to test that it works. That means I normally sync a smallish shelf. But, I have recently synced my "Read" shelf that has 818 books on it. There were no issues doing that.

Thinking about it, I don't think it will matter much. The API rate limiting is on total calls to the API, so it doesn't matter which direction the bulk of the data is going. Whether you have the books in calibre or not, the same information is being fetched when syncing a shelf from GR. It retrieves the shelf contents with what you see displayed in the syncing dialog. But, I would add the books by other methods with the GR identifier. The reason for that is it means you don't have to match the books from GR to calibre in the dialog. They should all be matched.

[UMNiK]

It got the full list without a hitch. Happily, I already had most of them, sadly, the goodreads ids for some are different from the ones pulled by the plugin, so I've some work ahead of me linking them up (luckily, the plugin makes it relatively painless). Thanks for the quick reply and for your work on the plugin!

[UMNiK]

Double posting to ask a new question.

Now that my Calibre and Goodreads are more or less in sync (a few straggler empty books to merge, but the find duplicates plugin will help me take care of those), what's the best way to keep them that way when it comes to adding new books? From what I understand, Goodreads has different ids for editions and the book overall and this plugin and goodreads metadata plugin pull the same "best edition" id unless ISBN comes into play, but the id when adding a book to a shelf on goodreads can be different depending on the edition.

Do I: a) Add whatever edition the site points me to to my massive shelf and sync it, which seems like rate limit abuse and likely to provide a dodgy id; b) Create a new empty book, copy-paste the title and author, and let the plugin add it to to-read, letting it pull whatever id it wants; c) Some other scenario I can't think of.

Essentially, I want the same book (meaning I don't care about the specific physical edition) to be shown as "Want to Read" (or other shelves) and exist in some form in Calibre in the quickest way possible that won't spawn duplicates in either place. Is there some way to get the "best" id from the book's page on Goodreads? Should I make a small temp shelf to sync from?

Apologies if I'm missing something obvious here.

[davidfor]

I do all the maintenance of books in calibre. When I get a new book, I add it to calibre and use the metadata download to get the metadata and identifiers for the books. And I add it to my "To Read" reading list and the reading lists for the devices I want it. I rarely put books in the "want-to-read" (or whatever it is called) shelf on GR.

As a rule, I don't worry that much about which edition is chosen from Goodreads. When I do (which usually means I am wasting time and should go read a book instead) I use the Goodreads Sync plugin to switch editions.

After that, when I start reading a book, I move it to my "Reading" reading list in calibre and use the plugin to add the book to the "currently reading" shelf and a genre shelf on GR. As I read the book, I use the update progress function to update GR. When I finish the book, the update progress function will set it as finished and set my rating (I rarely comment on the book) and put it on the "read" shelf. And I move the book to my "Finished" reading list.

A lot of the above is semi-automated. My Kobo Utilities plugin will fetch the reading status of books on the device when it is connected. And then call the update progress function after I accept those updates. That means two button clicks as I read plus selecting the rating when finished.

What that means is that I rarely fetch from GR. And mostly when testing the plugin. For a Kindle, I don't know the best way. I understand that there is some connection between the Kindle and GR, but, I have never used. But, if you just want to do an sync for a specific book, you can use the options under "Linked book". They upload and download options there should do the equivalent of the other options, but, just for that book.

[UMNiK]

Thanks for the detailed explanation of your workflow! Where you and I differ, it seems, is that I don't really want to have too many actual ebook files for books that I am not currently or imminently reading - what I'd like is to have a way to integrate my antilibrary (made up of empty books in Calibre that are now properly tagged as Goodreads.Want to Read and are on the default Goodreads to-read shelf, some with a native Goodreads note to the reason why) with my regular library (of books that do have files and some of which are read and marked as such both via calibre boolean column and with a Goodreads.Read tag and on goodreads' shelf). As mentioned before, the reasons for doing that are to be able to easily apply any hierarchical tags or reading lists and thus choose books based on multiple overlapping criteria (books of a certain genre with that award/nomination by these authors and so on, i.e. augmenting my personal notes with an external database). In other words, I use my lists to decide whether to get the book files rather than the other way around.

The in-built kindle-goodreads integration requires some adjustments for side-loaded books I do not care to do, nor do I need to track reading progress. All I want is to have a single shared entry for the same book inside Calibre and Goodreads that won't get messed up by any further metadata downloads (happily, the sync plugin and goodreads' exclusive shelves mean that any further operation on that book can be done in Calibre and easily synced to Goodreads). Can I just use the sync from (massive) Goodreads shelf to create the entry and disable identifier download in the Goodreads metadata plugin? Of course, it's great to have the option to manually change editions, but since they will all end up as a KFX file anyways, I'd just like to have a single, most stable id from the outset. My issue is I'm not sure whether adding the book to a shelf on goodreads will get that most stable id, and with no book file, the alternative is manually pasting in the title and author into an empty book and invoking the metadata download from Calibre, then synching the book to Goodreads's shelf, which is obviously a couple more pastes that I'd like (and as compared to the three clicks it takes to add the book to the shelf on Goodreads, then sync from shelf, then bulk metadata download).

UPDATE: After some testing and to be on the safe side, here's what I settled on:
1. Add the empty book to calibre by copy-pasting the title and author from goodreads (obviously, this will also let me see whether I've already shelved the book as ignored and any other notes).
2. Add that empty book to to-read using the Sync plugin and letting it find the best id match (which will also add the correct goodreads-related hierarchical tags).
3. Download book metadata, which should use that best id (at least for Goodreads and Goodreads More Tags (which has to be manually edited at the moment)).
4. Optionally follow the created link from Calibre sidebar to Goodreads (if closed the tab already) to write in any future notes.

This ensures that the best goodreads id is maintained for any further operations on the book. Ideally, there would be a userscript to make sure any Goodreads book opened in the browser opens to its "best edition", thus making it possible to add that to the shelf and sync from that to Calibre, but alas, no such thing exists AFAIK.

[isle9]

It's pretty weird how there isn't a shelf remove actions tab. When you remove a book from a Goodreads shelf, the tag still remains in Calibre.