[GUI Plugin] Goodreads Sync

[shadowdance]

i have actually no idea if that is a "good" or "proper" way, but after digging, i found a fix for me:

diff core.py core_orig.py 
219d218
<         request_token = {key.decode('utf-8'): value.decode('utf-8') for (key, value) in request_token.items()}
230d228
<         access_token= {key.decode('utf-8'): value.decode('utf-8') for (key, value) in access_token.items()}

[davidfor]

Originally Posted by shadowdance

hi,
new user of the plugin here..

When i try to authorize the plugin, I get above error...

calibre, version 4.99.4
ERROR: Unhandled exception: <b>KeyError</b>:'oauth_token'

calibre 4.99.4  embedded-python: False is64bit: True
Linux-5.4.17-towo.1-siduction-amd64-x86_64-with-debian-bullseye-sid Linux ('64bit', 'ELF')
('Linux', '5.4.17-towo.1-siduction-amd64', '#1 SMP PREEMPT siduction 5.4-17 (2020-02-03)')
Python 3.7.6
Linux: ('debian', 'bullseye/sid', '')
Interface language: None
Successfully initialized third party plugins: BOL_DE (0, 9, 0) && Goodreads Sync (1, 14, 1)
Traceback (most recent call last):
  File "calibre_plugins.goodreads_sync.config", line 1118, in authorize_plugin
  File "calibre_plugins.goodreads_sync.core", line 219, in get_request_token_secret
KeyError: 'oauth_token'

Any ideas on what i can do to fix that (apart learning python)?

I have to admit I haven't tried that with the Python 3 version of calibre. I will tonight.

But, did the GR page get displayed? And did you allow access from plugin? Not doing the latter would give an error, though it should get caught. Could you run calibre in debug mode and post or PM me the log? The logs will show details of the communications with GR and I should be able to see the issue.

[davidfor]

Originally Posted by shadowdance

i have actually no idea if that is a "good" or "proper" way, but after digging, i found a fix for me:

diff core.py core_orig.py 
219d218
<         request_token = {key.decode('utf-8'): value.decode('utf-8') for (key, value) in request_token.items()}
230d228
<         access_token= {key.decode('utf-8'): value.decode('utf-8') for (key, value) in access_token.items()}

Woops, didn't see the extra post on the next page.

That looks like another difference in how the different Python versions handle strings. I'll look at whether that is the right way to handle it.

[davidfor]

Originally Posted by shadowdance

i have actually no idea if that is a "good" or "proper" way, but after digging, i found a fix for me:

diff core.py core_orig.py 
219d218
<         request_token = {key.decode('utf-8'): value.decode('utf-8') for (key, value) in request_token.items()}
230d228
<         access_token= {key.decode('utf-8'): value.decode('utf-8') for (key, value) in access_token.items()}

I have implemented a fix based on this. I have asked for the release version to be updated with this.

[shadowdance]

Originally Posted by davidfor

I have implemented a fix based on this. I have asked for the release version to be updated with this.

wow - that was quick!
thank you!

it does work with the new version 1.14.2

[Orz]

My primary source for books is Amazon. Kindle books from Amazon do not have ISBNs, and Goodreads can work much better with Amazon's ASIN when searching and matching, etc.

When performing a batch search, I find that the plugin never matches anything automatically. I had to manually search and select for thousands of books. Is it possible to support or default to using ASIN as the relational database primary key?

Also, it would be really helpful to support "fuzzy search" when matching Goodreads books to Calibre books, then just present a table of fuzzy searched results for confirmation if it can have secondary options in a dropdown menu, that would be even better.

[davidfor]

Originally Posted by Orz

My primary source for books is Amazon. Kindle books from Amazon do not have ISBNs, and Goodreads can work much better with Amazon's ASIN when searching and matching, etc.

When performing a batch search, I find that the plugin never matches anything automatically. I had to manually search and select for thousands of books. Is it possible to support or default to using ASIN as the relational database primary key?

Also, it would be really helpful to support "fuzzy search" when matching Goodreads books to Calibre books, then just present a table of fuzzy searched results for confirmation if it can have secondary options in a dropdown menu, that would be even better.

For most book matching, I recommend using the metadata download, and the Goodreads metadata source plugin. That can match on ASIN as well as ISBN and author/title.

Adding ASIN matching here would be possible. But, it would mean duplicating code from the metadata source plugin. And that means maintaining it in two places. I don't really see an advantage in doing it this way, so I don't really think it is worth the extra work.

[Rellwood]

Ok, the updated plugin is a bit faster. About 25 seconds per hundred books synced. However, that is tested on a library that has a small .db file. about 15MB. I haven't determinined if it is a matter of it being a large database or if it has to do with having a lot of tags. The rules for sync don't make a difference.

[Rellwood]

Originally Posted by davidfor

Now, if the progress bar takes time to update, it is because the process behind it is taking time to update. Before I added the progress bar, calibre gave no indication that anything was happening. It was basically hung. The progress bar updates are just added at points in the loop of progress between pages or books being changed.

Nope. I contest that it takes longer.

I know that Calibre hangs while it updates the books via sync with the old plugin, however It took me 8:27 to sync my read shelf using the new plugin. There are 1162 books.

With the older plugin it took :25.94. I know this because I set the stopwatch to start when the sync box closed out and stop when the arrow icon turned into a finger when it was hovering over the Goodreads button on the menu bar.

Also, when I say update page, because the progress bar is so slow I am able to see it literally say “page 1”……….”page 2”…………”page 3” and so on. I may have saig "Updating Page 1" but I think I meant "Page 1" and each page represents a book.

I am not making this up, and I am not “perceiving” it to be slower. Why would I complain for months about this if I was just making perception errors that I could easily clear up with a stopwatch?

I have many times brought this up here. I understand that this is a problem that only affects me because no one would have been quiet if they were dealing with this. Also, I don't appreciate your being condescending. I know that this is a hard problem to explain, and I have tried to explain it, but since it only happens to me it sounds like I am being nit-picky, but I am not.

This is a great plugin, I love it, and I have contributed to it in the past. (I think), it is an integral part of my Calibre library. So having it work right is important. I don't doubt for a minute that you made the plugin fine, but for whatever reason it isn't playing nice with my library.

[Rellwood]

Could having two versions of Calibre be a reason? I have the 32bit and the 64bit. Could having both versions be a reason why the plugin is slow? Is there a way outside of removing Calibre from my coputer to test it?


No. I removed the older Calibre and it still runs a bit slower. About 100 books per 30 seconds.

[soundium]

Hello,

I am getting a strange error if I try to link my books to Goodreads.

calibre, version 4.13.0
ERROR: Unhandled exception: <b>IndexError</b>:list index out of range

calibre 4.13  embedded-python: True is64bit: True
Darwin-19.4.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '19.4.0', 'Darwin Kernel Version 19.4.0: Wed Mar  4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64')
Python 2.7.16
OSX: ('10.15.4', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: Goodreads More Tags (1, 1, 0) && Find Duplicates (1, 6, 3) && Goodreads Sync (1, 14, 2) && Goodreads_Rating (3, 0, 0) && Goodreads (1, 4, 0)
Traceback (most recent call last):
  File "calibre_plugins.goodreads_sync.action", line 458, in search_goodreads_to_link_book
  File "calibre_plugins.goodreads_sync.action", line 504, in search_to_link_books
  File "calibre_plugins.goodreads_sync.dialogs", line 1096, in selected_goodreads_book
  File "calibre_plugins.goodreads_sync.dialogs", line 971, in selected_goodreads_book
IndexError: list index out of range

What should I do to resolve this issue?

[davidfor]

Originally Posted by soundium

Hello,

I am getting a strange error if I try to link my books to Goodreads.

calibre, version 4.13.0
ERROR: Unhandled exception: <b>IndexError</b>:list index out of range

calibre 4.13  embedded-python: True is64bit: True
Darwin-19.4.0-x86_64-i386-64bit Darwin ('64bit', '')
('Darwin', '19.4.0', 'Darwin Kernel Version 19.4.0: Wed Mar  4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64')
Python 2.7.16
OSX: ('10.15.4', ('', '', ''), 'x86_64')
Interface language: None
Successfully initialized third party plugins: Goodreads More Tags (1, 1, 0) && Find Duplicates (1, 6, 3) && Goodreads Sync (1, 14, 2) && Goodreads_Rating (3, 0, 0) && Goodreads (1, 4, 0)
Traceback (most recent call last):
  File "calibre_plugins.goodreads_sync.action", line 458, in search_goodreads_to_link_book
  File "calibre_plugins.goodreads_sync.action", line 504, in search_to_link_books
  File "calibre_plugins.goodreads_sync.dialogs", line 1096, in selected_goodreads_book
  File "calibre_plugins.goodreads_sync.dialogs", line 971, in selected_goodreads_book
IndexError: list index out of range

What should I do to resolve this issue?

What was the actual steps that did this? I can get to that error if use the "Link to Goodreads" function for a book that doesn't find a result on Goodreads and then press the "Switch Edition" button. But, the stack dump I get isn't the same.

[soundium]

Originally Posted by davidfor

What was the actual steps that did this? I can get to that error if use the "Link to Goodreads" function for a book that doesn't find a result on Goodreads and then press the "Switch Edition" button. But, the stack dump I get isn't the same.

I marked five books (qty does not play a role) and used the "Link to Goodreads" function if the book does not find the result I clicked OK. Always the same result - this error.

I think this plugin needs some improvement:
- if the book finds result -> click "Approve" or "Reject"
- if the book does not find any -> click "Skip"
- If you checked all books -> click "OK".

[davidfor]

Originally Posted by soundium

I marked five books (qty does not play a role) and used the "Link to Goodreads" function if the book does not find the result I clicked OK. Always the same result - this error.

I think this plugin needs some improvement:
- if the book finds result -> click "Approve" or "Reject"
- if the book does not find any -> click "Skip"
- If you checked all books -> click "OK".

The "OK" button is what you are calling "Approve" and the "Skip" button is what you are calling "Reject" (look at the tooltip for the "Skip" button). The problem is that the "OK" button should be disabled when the search doesn't find anything. Or it should handle this gracefully. Same with the "Switch Edition" button. There isn't an "Accept all" button as unless all searches only found one book, this would be dangerous.

I'll fix this, but, in the meantime, just don't press "OK" when there are no books matched.

[soundium]

Originally Posted by davidfor

The "OK" button is what you are calling "Approve" and the "Skip" button is what you are calling "Reject" (look at the tooltip for the "Skip" button). The problem is that the "OK" button should be disabled when the search doesn't find anything. Or it should handle this gracefully. Same with the "Switch Edition" button. There isn't an "Accept all" button as unless all searches only found one book, this would be dangerous.

I'll fix this, but, in the meantime, just don't press "OK" when there are no books matched.

Thank you for the explanation. In my opinion, it would be elementary for a user to use the same UI as Calibre UI - the same approach as macOS etc. Different UI make it user-unfriendly.
Thank you for your support and fixing the issue.