Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre > Plugins

Notices

Reply
 
Thread Tools Search this Thread
Old 12-15-2015, 07:43 PM   #241
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 8,515
Karma: 11052666
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo: Touch, Glo, Aura H2O, Glo HD
Quote:
Originally Posted by Krazykiwi View Post
Could you make it add any random character at the end? a %s.%random-alpha-char% or %s-%randomalphachar ('scuse my lack of python-fu, but that ought to be a fairly simple little function even if it's not a built-in right?) That would make every request be treated as "the first time".
The fact that the URL works with a dash or some other character after the id is in my minds a bug in the site. If they fix this and start returning an error when the characters after the id are not what is expected, what do you do? Or, just as likely, fix the code so that all variations of the URL for a book are treated the same.

But, my real problem with doing this is being a "good citizen". The proper URL has been blocked for a reason. We should respect that. Maybe we should work out why it was blocked in the first place.
davidfor is offline   Reply With Quote
Old 12-15-2015, 11:18 PM   #242
Krazykiwi
Enthusiast
Krazykiwi could sell banana peel slippers to a Deveel.Krazykiwi could sell banana peel slippers to a Deveel.Krazykiwi could sell banana peel slippers to a Deveel.Krazykiwi could sell banana peel slippers to a Deveel.Krazykiwi could sell banana peel slippers to a Deveel.Krazykiwi could sell banana peel slippers to a Deveel.Krazykiwi could sell banana peel slippers to a Deveel.Krazykiwi could sell banana peel slippers to a Deveel.Krazykiwi could sell banana peel slippers to a Deveel.Krazykiwi could sell banana peel slippers to a Deveel.Krazykiwi could sell banana peel slippers to a Deveel.
 
Posts: 29
Karma: 3088
Join Date: Jan 2013
Device: Too many random androids to list
I'm not so sure it's hurting anything.

The URL's working with text after the ID is definitely by design, it's not a bug. The url's internally on the site, such as the ones you get when you use the search interface or add a link into one of their forum posts using that search (an entirely different one to the main site search), are constructed by contatenating/truncating the title something similar to how Calibre shortens titles on save to disk. You can see them changing when you edit a book title (or, since Authors work the same way, if you edit an author name on the site.) It's been that way for years too, and is unlikely to be changing .

Secondly, if the GR plugin was truly being a good citizen, it'd be using the API, not scraping.

The API certainly has rate limits set (And doesn't the UI stop you pulling up more than 50 books at a time in some circumstances? It's been a very long time since I had 50 books to add metadata too, so I may be misremembering.)

I think, since Calibre is already being a bit of a dandy highwayman as far as data fetching goes, this isn't so bad. As long as it's not hitting more than once a second, it's not being any worse of a citizen than it's ever been.
Krazykiwi is offline   Reply With Quote
Old 12-17-2015, 03:06 AM   #243
trying
Member
trying began at the beginning.
 
Posts: 15
Karma: 10
Join Date: Oct 2013
Device: none
Quote:
Originally Posted by Krazykiwi View Post
Could you make it add any random character at the end? a %s.%random-alpha-char% or %s-%randomalphachar ('scuse my lack of python-fu, but that ought to be a fairly simple little function even if it's not a built-in right?) That would make every request be treated as "the first time".
Sure, just do something like:
Code:
import random
randomletter = chr(random.randint(97,122))
And you could even randomly add a random # of lowercase letters, say 5-10.

But it's not necessary at this point. Just adding a dash works fine, and I also doubt goodreads will change this (for the same reasons you gave in your next post).

Without my one char fix, however, people will see 403 errors when they try to download goodreads metadata for books that already have a "goodreads:" identifier. This behavior is easily reproducible by anyone.

Quote:
Originally Posted by Krazykiwi View Post
I'm not so sure it's hurting anything.

The URL's working with text after the ID is definitely by design, it's not a bug. The url's internally on the site, such as the ones you get when you use the search interface or add a link into one of their forum posts using that search (an entirely different one to the main site search), are constructed by contatenating/truncating the title something similar to how Calibre shortens titles on save to disk. You can see them changing when you edit a book title (or, since Authors work the same way, if you edit an author name on the site.) It's been that way for years too, and is unlikely to be changing .

Secondly, if the GR plugin was truly being a good citizen, it'd be using the API, not scraping.

The API certainly has rate limits set (And doesn't the UI stop you pulling up more than 50 books at a time in some circumstances? It's been a very long time since I had 50 books to add metadata too, so I may be misremembering.)

I think, since Calibre is already being a bit of a dandy highwayman as far as data fetching goes, this isn't so bad. As long as it's not hitting more than once a second, it's not being any worse of a citizen than it's ever been.
I completely agree with everything said.

I presume (though I didn't look into it) that Calibre is purposely not sending more than 1 request per second per Calibre instance in order to try to follow the spirit of the Goodreads terms of use if not the letter.

If the plugin did use the API then the limit would be 1 request per second per developer key. All users of the plugin would then have to get their own key, otherwise there could only be 1 request per second for all simultaneous users of the plugin.
trying is offline   Reply With Quote
Old 12-17-2015, 08:47 AM   #244
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 8,515
Karma: 11052666
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo: Touch, Glo, Aura H2O, Glo HD
Quote:
Originally Posted by trying View Post
I presume (though I didn't look into it) that Calibre is purposely not sending more than 1 request per second per Calibre instance in order to try to follow the spirit of the Goodreads terms of use if not the letter.
No, calibres request limit has nothing to do with Goodreads. It is the chosen limit for all metadata source queries. If it was chosen for a particular site, it will be one of built-in sites.
Quote:
If the plugin did use the API then the limit would be 1 request per second per developer key. All users of the plugin would then have to get their own key, otherwise there could only be 1 request per second for all simultaneous users of the plugin.
The idea of every user of the plugin needing their own developer key is pretty bad. Hopefully it won't come to that.

Personally, I'm not interested in working around Goodreads restrictions. The real question on all this is why is the 403 error appearing? I haven't seen anything from Goodreads on restrictions like this. The only references I can see are some problems when using the API. I also haven't seen the error myself. And that includes when I am fiddling with a metadata source plugin and continuously repeating the same request.
davidfor is offline   Reply With Quote
Old 12-21-2015, 03:57 PM   #245
Freakeao
Junior Member
Freakeao began at the beginning.
 
Posts: 4
Karma: 10
Join Date: Nov 2012
Device: none
Quote:
Originally Posted by BetterRed View Post
I think it's it's something to do with timezones, I'm not aware of any setting you can adjust and I don't understand the 'why is it so'.

BR
This is still an issue for me. Is there any fix or work around for this?

Oops. didn't quote quite the right post.

Published dates are one day off. This can result in being a whole year off. Jan 1 2005->Dec 31 2004
Freakeao is offline   Reply With Quote
Old Yesterday, 05:00 PM   #246
BookJunkieLI
Addict
BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.
 
BookJunkieLI's Avatar
 
Posts: 363
Karma: 572984
Join Date: Jan 2010
Location: Long Island
Device: Nook, REB 1100, Ebookwise 1500, Jb-Lite, PRS-650, PB302, Asus T-90MT
Has anyone been having a problem with this plugin lately? Every time I go to download metadata for a book in the last couple of days if I don't have an ISBN number it returns with a no match error message. Yet if I go to Goodreads and do the same exact search it will bring up the book I'm looking for. I checked and I am running the latest version of the plugin.
BookJunkieLI is offline   Reply With Quote
Old Yesterday, 06:44 PM   #247
Abusor
Junior Member
Abusor began at the beginning.
 
Posts: 2
Karma: 10
Join Date: Nov 2013
Device: Android
Quote:
Originally Posted by BookJunkieLI View Post
Has anyone been having a problem with this plugin lately? Every time I go to download metadata for a book in the last couple of days if I don't have an ISBN number it returns with a no match error message. Yet if I go to Goodreads and do the same exact search it will bring up the book I'm looking for. I checked and I am running the latest version of the plugin.
Yup. I've noticed the same.
Abusor is offline   Reply With Quote
Old Yesterday, 08:53 PM   #248
davidfor
Grand Sorcerer
davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.davidfor ought to be getting tired of karma fortunes by now.
 
Posts: 8,515
Karma: 11052666
Join Date: Jul 2011
Location: Sydney, Australia
Device: Kobo: Touch, Glo, Aura H2O, Glo HD
There has been a small change in the search results page that broke it. The attached version should fix it. Please test and tell me so I can arrange for it to be released.
Attached Files
File Type: zip Goodreads-1.1.11.zip (21.6 KB, 2 views)
davidfor is offline   Reply With Quote
Old Today, 01:17 AM   #249
BookJunkieLI
Addict
BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.BookJunkieLI ought to be getting tired of karma fortunes by now.
 
BookJunkieLI's Avatar
 
Posts: 363
Karma: 572984
Join Date: Jan 2010
Location: Long Island
Device: Nook, REB 1100, Ebookwise 1500, Jb-Lite, PRS-650, PB302, Asus T-90MT
Yay! I tried it on a few different books and that seems to have done the trick. Thank you!
BookJunkieLI is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[GUI Plugin] Goodreads Sync kiwidude Plugins 808 01-18-2016 09:27 AM
[Metadata Download Plugin] Goodreads Metadata **Deprecated** kiwidude Plugins 30 04-23-2011 03:10 PM
[Covers Plugin] Goodreads Covers **Deprecated** kiwidude Plugins 13 04-17-2011 06:09 PM
metadata plugin redneck_momma Plugins 1 05-21-2010 09:41 PM


All times are GMT -4. The time now is 02:10 AM.


MobileRead.com is a privately owned, operated and funded community.