Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 12-15-2015, 06:43 PM   #241
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by Krazykiwi View Post
Could you make it add any random character at the end? a %s.%random-alpha-char% or %s-%randomalphachar ('scuse my lack of python-fu, but that ought to be a fairly simple little function even if it's not a built-in right?) That would make every request be treated as "the first time".
The fact that the URL works with a dash or some other character after the id is in my minds a bug in the site. If they fix this and start returning an error when the characters after the id are not what is expected, what do you do? Or, just as likely, fix the code so that all variations of the URL for a book are treated the same.

But, my real problem with doing this is being a "good citizen". The proper URL has been blocked for a reason. We should respect that. Maybe we should work out why it was blocked in the first place.
davidfor is offline   Reply With Quote
Old 12-15-2015, 10:18 PM   #242
Krazykiwi
Zealot
Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.Krazykiwi ought to be getting tired of karma fortunes by now.
 
Posts: 137
Karma: 2156958
Join Date: Jan 2013
Device: Too many random androids to list
I'm not so sure it's hurting anything.

The URL's working with text after the ID is definitely by design, it's not a bug. The url's internally on the site, such as the ones you get when you use the search interface or add a link into one of their forum posts using that search (an entirely different one to the main site search), are constructed by contatenating/truncating the title something similar to how Calibre shortens titles on save to disk. You can see them changing when you edit a book title (or, since Authors work the same way, if you edit an author name on the site.) It's been that way for years too, and is unlikely to be changing .

Secondly, if the GR plugin was truly being a good citizen, it'd be using the API, not scraping.

The API certainly has rate limits set (And doesn't the UI stop you pulling up more than 50 books at a time in some circumstances? It's been a very long time since I had 50 books to add metadata too, so I may be misremembering.)

I think, since Calibre is already being a bit of a dandy highwayman as far as data fetching goes, this isn't so bad. As long as it's not hitting more than once a second, it's not being any worse of a citizen than it's ever been.
Krazykiwi is offline   Reply With Quote
Advert
Old 12-17-2015, 02:06 AM   #243
trying
Member
trying doesn't littertrying doesn't litter
 
Posts: 21
Karma: 104
Join Date: Oct 2013
Device: none
Quote:
Originally Posted by Krazykiwi View Post
Could you make it add any random character at the end? a %s.%random-alpha-char% or %s-%randomalphachar ('scuse my lack of python-fu, but that ought to be a fairly simple little function even if it's not a built-in right?) That would make every request be treated as "the first time".
Sure, just do something like:
Code:
import random
randomletter = chr(random.randint(97,122))
And you could even randomly add a random # of lowercase letters, say 5-10.

But it's not necessary at this point. Just adding a dash works fine, and I also doubt goodreads will change this (for the same reasons you gave in your next post).

Without my one char fix, however, people will see 403 errors when they try to download goodreads metadata for books that already have a "goodreads:" identifier. This behavior is easily reproducible by anyone.

Quote:
Originally Posted by Krazykiwi View Post
I'm not so sure it's hurting anything.

The URL's working with text after the ID is definitely by design, it's not a bug. The url's internally on the site, such as the ones you get when you use the search interface or add a link into one of their forum posts using that search (an entirely different one to the main site search), are constructed by contatenating/truncating the title something similar to how Calibre shortens titles on save to disk. You can see them changing when you edit a book title (or, since Authors work the same way, if you edit an author name on the site.) It's been that way for years too, and is unlikely to be changing .

Secondly, if the GR plugin was truly being a good citizen, it'd be using the API, not scraping.

The API certainly has rate limits set (And doesn't the UI stop you pulling up more than 50 books at a time in some circumstances? It's been a very long time since I had 50 books to add metadata too, so I may be misremembering.)

I think, since Calibre is already being a bit of a dandy highwayman as far as data fetching goes, this isn't so bad. As long as it's not hitting more than once a second, it's not being any worse of a citizen than it's ever been.
I completely agree with everything said.

I presume (though I didn't look into it) that Calibre is purposely not sending more than 1 request per second per Calibre instance in order to try to follow the spirit of the Goodreads terms of use if not the letter.

If the plugin did use the API then the limit would be 1 request per second per developer key. All users of the plugin would then have to get their own key, otherwise there could only be 1 request per second for all simultaneous users of the plugin.
trying is offline   Reply With Quote
Old 12-17-2015, 07:47 AM   #244
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Quote:
Originally Posted by trying View Post
I presume (though I didn't look into it) that Calibre is purposely not sending more than 1 request per second per Calibre instance in order to try to follow the spirit of the Goodreads terms of use if not the letter.
No, calibres request limit has nothing to do with Goodreads. It is the chosen limit for all metadata source queries. If it was chosen for a particular site, it will be one of built-in sites.
Quote:
If the plugin did use the API then the limit would be 1 request per second per developer key. All users of the plugin would then have to get their own key, otherwise there could only be 1 request per second for all simultaneous users of the plugin.
The idea of every user of the plugin needing their own developer key is pretty bad. Hopefully it won't come to that.

Personally, I'm not interested in working around Goodreads restrictions. The real question on all this is why is the 403 error appearing? I haven't seen anything from Goodreads on restrictions like this. The only references I can see are some problems when using the API. I also haven't seen the error myself. And that includes when I am fiddling with a metadata source plugin and continuously repeating the same request.
davidfor is offline   Reply With Quote
Old 12-21-2015, 02:57 PM   #245
Freakeao
Connoisseur
Freakeao began at the beginning.
 
Posts: 51
Karma: 10
Join Date: Nov 2012
Device: none
Quote:
Originally Posted by BetterRed View Post
I think it's it's something to do with timezones, I'm not aware of any setting you can adjust and I don't understand the 'why is it so'.

BR
This is still an issue for me. Is there any fix or work around for this?

Oops. didn't quote quite the right post.

Published dates are one day off. This can result in being a whole year off. Jan 1 2005->Dec 31 2004
Freakeao is offline   Reply With Quote
Advert
Old 02-06-2016, 04:00 PM   #246
BookJunkieLI
Evangelist
BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.
 
BookJunkieLI's Avatar
 
Posts: 435
Karma: 572984
Join Date: Jan 2010
Location: Long Island
Device: Kobo Libra 2, Kindle 4, Nook Gl4, Nook STR, REB 1100, Ebookwise 1500,
Has anyone been having a problem with this plugin lately? Every time I go to download metadata for a book in the last couple of days if I don't have an ISBN number it returns with a no match error message. Yet if I go to Goodreads and do the same exact search it will bring up the book I'm looking for. I checked and I am running the latest version of the plugin.
BookJunkieLI is offline   Reply With Quote
Old 02-06-2016, 05:44 PM   #247
Abusor
Junior Member
Abusor began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Nov 2013
Device: Android
Quote:
Originally Posted by BookJunkieLI View Post
Has anyone been having a problem with this plugin lately? Every time I go to download metadata for a book in the last couple of days if I don't have an ISBN number it returns with a no match error message. Yet if I go to Goodreads and do the same exact search it will bring up the book I'm looking for. I checked and I am running the latest version of the plugin.
Yup. I've noticed the same.
Abusor is offline   Reply With Quote
Old 02-06-2016, 07:53 PM   #248
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
There has been a small change in the search results page that broke it. The attached version should fix it. Please test and tell me so I can arrange for it to be released.
Attached Files
File Type: zip Goodreads-1.1.11.zip (21.6 KB, 266 views)
davidfor is offline   Reply With Quote
Old 02-07-2016, 12:17 AM   #249
BookJunkieLI
Evangelist
BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.
 
BookJunkieLI's Avatar
 
Posts: 435
Karma: 572984
Join Date: Jan 2010
Location: Long Island
Device: Kobo Libra 2, Kindle 4, Nook Gl4, Nook STR, REB 1100, Ebookwise 1500,
Yay! I tried it on a few different books and that seems to have done the trick. Thank you!
BookJunkieLI is offline   Reply With Quote
Old 02-08-2016, 06:43 AM   #250
amely
Junior Member
amely began at the beginning.
 
Posts: 6
Karma: 10
Join Date: Oct 2015
Device: none
Thank you davidfor I tried it on about 40 books and it works perfectly both in single and bulk metadata download.
amely is offline   Reply With Quote
Old 02-08-2016, 02:19 PM   #251
madeleinelevy
Enthusiast
madeleinelevy began at the beginning.
 
Posts: 35
Karma: 10
Join Date: Jun 2014
Device: Paperwhite
It works. Thanks,
madeleinelevy is offline   Reply With Quote
Old 02-08-2016, 02:47 PM   #252
kiwidude
calibre/Sigil Developer
kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.kiwidude ought to be getting tired of karma fortunes by now.
 
Posts: 4,601
Karma: 2092290
Join Date: Oct 2010
Location: Australia
Device: Kindle Oasis
v1.1.11 Released

Changes in this release
  • Site changes for the description/comments.
  • Site and option changes for genre and classification.

Thanks to davidfor for making the changes.
kiwidude is offline   Reply With Quote
Old 02-14-2016, 07:12 PM   #253
elricks
Long in the tooth junior
elricks began at the beginning.
 
Posts: 13
Karma: 10
Join Date: Apr 2011
Location: melbourne, australia
Device: none
Good morning.

I am using the Goodreads Metadata Plugin, and I have an error in the mapping function.

I originally reported this to the main bug system - https://bugs.launchpad.net/calibre/+bug/1540848, but was referred to this forum, as it appears the 'bug' is in the plugin. Full details of the circumstances are posted in the original bug report.

A high level summary - In Preferences - Adding Books - Create rules to filter/transform tags.

I have set up some rules:

An Example - 'Replace the tag, if it matches pattern: biog*
with the tag: Biography

When I run 'download metadata in bulk' the above tag ends up biographyphyphyphpyphyp, and variants. Some of the residual original tag info is added to the end of the long line, no spaces are copied or added.

I am now using Calibre 2.51, Windows 7 64 bit. In my original report, I was using 2.50.1

Thanks - shirley
elricks is offline   Reply With Quote
Old 02-16-2016, 07:06 AM   #254
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 24,908
Karma: 47303748
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo:Touch,Glo, AuraH2O, GloHD,AuraONE, ClaraHD, Libra H2O; tolinoepos
Shirley: It isn't the plugin. I did some testing using the tag mapper test dialog. I had the same results. I've added that to the bug report.
davidfor is offline   Reply With Quote
Old 02-16-2016, 06:30 PM   #255
whitearrow
Guru
whitearrow ought to be getting tired of karma fortunes by now.whitearrow ought to be getting tired of karma fortunes by now.whitearrow ought to be getting tired of karma fortunes by now.whitearrow ought to be getting tired of karma fortunes by now.whitearrow ought to be getting tired of karma fortunes by now.whitearrow ought to be getting tired of karma fortunes by now.whitearrow ought to be getting tired of karma fortunes by now.whitearrow ought to be getting tired of karma fortunes by now.whitearrow ought to be getting tired of karma fortunes by now.whitearrow ought to be getting tired of karma fortunes by now.whitearrow ought to be getting tired of karma fortunes by now.
 
Posts: 808
Karma: 2260766
Join Date: Apr 2008
Device: Kindle Oasis 2
Is there a way the plugin could include the "I own this book" field on Goodreads?
whitearrow is offline   Reply With Quote
Reply

Tags
goodreads, metadata

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Goodreads Sync kiwidude Plugins 1713 03-16-2024 11:37 PM
[Metadata Download Plugin] Goodreads Metadata **Deprecated** kiwidude Plugins 30 04-23-2011 02:10 PM
[Covers Plugin] Goodreads Covers **Deprecated** kiwidude Plugins 13 04-17-2011 05:09 PM
metadata plugin redneck_momma Plugins 1 05-21-2010 08:41 PM


All times are GMT -4. The time now is 10:07 AM.


MobileRead.com is a privately owned, operated and funded community.