Possible bug causing Amazon captcha? - Page 3

kovidgoyal · 02-11-2017, 08:37 PM

Hmm, well I changed the plugin to use a random user agent, that might help for a while, although as I said before, this can easily become an endless arms race, so dont rely on this plugin working for ever.

SteveB23 · 02-11-2017, 08:41 PM

I'm not saying it's a bug, and Amazon is clearly within its rights to update the way it manages metadata requests. But Calibre has always, as long as I've been using it, included the ability download metadata. I'm just offering my feedback (as a very grateful user) that I really like that feature and I'm sad that it is no longer able to perform its function, in the hope that somehow it can be restored. I understand and agree with Kovid's concern that this appears to be an escalating arms war with Amazon, and I don't know what the answer might be. There have been a couple of suggestions that have been rejected: 1) pass to the CAPTCHA request to the user and allow them to respond, or 2) allow users to specify or override the UA setting. Maybe it's time to reconsider one or both of them?

TSRV · 02-11-2017, 09:50 PM

Quote:

Originally Posted by PeterT

Just wondering why you think it's a calibre bug, when it's the website (Amazon) deciding to requiring a captcha to validate a human is accessing the data as opposed to an automated process?

It doesn't matter whose bug it is. It is impacting the ease of use within Calibre, and it did in fact work before.

It seems like the whole "fetch metadata" feature will have to be discontinued if this bug isn't fixed, because just about all of the searches I do on non-Amazon sites come up with no metadata found. When no site works, it makes it look a whole lot more like a Calibre bug.

Quote:

Originally Posted by kovidgoyal

Hmm, well I changed the plugin to use a random user agent, that might help for a while, although as I said before, this can easily become an endless arms race, so dont rely on this plugin working for ever.

Would this be reflected in the latest version of Calibre? If so, it's not working now.

If the arms race is not a viable option, perhaps we could explore other databases that may have similar abilities to index books as Amazon does? Literally none of the databases are working for me right now, on books that are still in-print in the US market.

kovidgoyal · 02-11-2017, 10:19 PM

No, the next version. And getting metadata from google books works fine for me. Not to mention the dozens of other metadata sources available via Get plugins.

TSRV · 02-12-2017, 12:45 AM

Quote:

Originally Posted by kovidgoyal

No, the next version. And getting metadata from google books works fine for me. Not to mention the dozens of other metadata sources available via Get plugins.

After getting a few plugins it seems to be working for me from other sources. I tested about 10 books on Google before posting and kept getting 0 results, but I think perhaps the error was that the subtitle was included, and when I get rid of that, Google finds it (and then replaces it with the exact same title:subtitle).

Thank you for your assistance and continued development.

SteveB23 · 02-12-2017, 03:35 PM

What plug-ins are we talking about here, and what sources do they use? (I'm pretty much using just the defaults.) One of the metadata fields I like getting from Amazon is their rating system, and other than very occasional ratings from Google, I don't believe I see that from any other sources.

kovidgoyal · 02-12-2017, 04:57 PM

You get plugins from Preferences->Get new plugins and IIRC there are several other plugins that have ratings, for example, goodreads.

Oddball · 02-24-2017, 06:13 PM

The way this plugin operates reminds me of the way the RIAA beats up on their own customers. Amazon is just doing the same thing. And this WILL cost them the same way it cost the RIAA in big money and loss of market share. The MASSIVE change done to amazon ebooks to the kfx and kf8 files formats is just on more of example of a company dong its damnedest to shoot itself in the head (just like Coca-cola with New Coke). I guess bezos has forgotten that under the Sony FAIR USE that a backup of digital media is legal.. all i see this is preventing that. Enough of that, since the 2.80 update I get the Captcha error after ONE metadata download...AMAZON SUCKS.

kovidgoyal · 02-25-2017, 09:55 AM

Probably time for me to look into replacing the amazon builtin plugin with some other site. Given the way amazon has been behaving of late, the less things in calibre that depend on it the happier I'll be.

Any recommendations from people that regularly use other general purpose metadata sources?

theducks · 02-25-2017, 12:42 PM

I use both Fantastic Fiction sites (I am over 21... by a long shot

)
I still get "Not Founds" (mostly for self pub found at Amazon USA), so something else is needed to fll those gaps

Purple Lady · 02-25-2017, 04:43 PM

Quote:

Originally Posted by theducks

I use both Fantastic Fiction sites (I am over 21... by a long shot

)
I still get "Not Founds" (mostly for self pub found at Amazon USA), so something else is needed to fll those gaps

Will self pub that is a part of Kindle unlimited be anywhere other than Amazon? They have to be exclusive with Amazon.

kovidgoyal · 02-26-2017, 04:47 AM

There are two fantastic fiction sites? I'm only familiar with one: www.fantasticfiction.com and as far as I know, it focuses only on relatively recent books in genres like SF, Romance, Mystery etc. I dont know how suitable it would be as a general purpose metadata source. Looking at its homepage it says it has about half a million books. I suppose that's not too bad, it's about a quarter of amazon's catalog.

theducks · 02-26-2017, 10:52 AM

Quote:

Originally Posted by kovidgoyal

There are two fantastic fiction sites? I'm only familiar with one: www.fantasticfiction.com and as far as I know, it focuses only on relatively recent books in genres like SF, Romance, Mystery etc. I dont know how suitable it would be as a general purpose metadata source. Looking at its homepage it says it has about half a million books. I suppose that's not too bad, it's about a quarter of amazon's catalog.

Yes, there is FF Adult for those over NN

There are now 2 metadata Plugins

kovidgoyal · 02-26-2017, 01:19 PM

How does one get to the adult version? I can see no mention of it on the FF home page and googling it returns nothing useful.

itimpi · 02-26-2017, 01:46 PM

Quote:

Originally Posted by kovidgoyal

How does one get to the adult version? I can see no mention of it on the FF home page and googling it returns nothing useful.

it is at https://www.ffadultsonly.com/

You tend to get taken there automatically from the main site if you search for an author/book that is listed on the adult site.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Aussies launch anti-CAPTCHA petition. Is it time to kill CAPTCHAs?	Alexander Turcic	Lounge	30	08-16-2013 07:32 PM
Bug in Kobo processing of epub files causing hang in "Processing content"	BensonBear	Kobo Reader	21	12-21-2012 05:47 AM
Get books bug for Amazon UK	rustleg	Library Management	1	10-21-2012 01:25 PM

02-11-2017, 08:37 PM	#31
kovidgoyal creator of calibre Posts: 45,221 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Hmm, well I changed the plugin to use a random user agent, that might help for a while, although as I said before, this can easily become an endless arms race, so dont rely on this plugin working for ever.

02-11-2017, 08:41 PM	#32
SteveB23 Junior Member Posts: 6 Karma: 10 Join Date: Jul 2016 Device: Kindle Apps (Desktop, Android phone & tablet)	I'm not saying it's a bug, and Amazon is clearly within its rights to update the way it manages metadata requests. But Calibre has always, as long as I've been using it, included the ability download metadata. I'm just offering my feedback (as a very grateful user) that I really like that feature and I'm sad that it is no longer able to perform its function, in the hope that somehow it can be restored. I understand and agree with Kovid's concern that this appears to be an escalating arms war with Amazon, and I don't know what the answer might be. There have been a couple of suggestions that have been rejected: 1) pass to the CAPTCHA request to the user and allow them to respond, or 2) allow users to specify or override the UA setting. Maybe it's time to reconsider one or both of them?

02-11-2017, 10:19 PM	#34
kovidgoyal creator of calibre Posts: 45,221 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	No, the next version. And getting metadata from google books works fine for me. Not to mention the dozens of other metadata sources available via Get plugins.

02-12-2017, 03:35 PM	#36
SteveB23 Junior Member Posts: 6 Karma: 10 Join Date: Jul 2016 Device: Kindle Apps (Desktop, Android phone & tablet)	What plug-ins are we talking about here, and what sources do they use? (I'm pretty much using just the defaults.) One of the metadata fields I like getting from Amazon is their rating system, and other than very occasional ratings from Google, I don't believe I see that from any other sources.

02-12-2017, 04:57 PM	#37
kovidgoyal creator of calibre Posts: 45,221 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	You get plugins from Preferences->Get new plugins and IIRC there are several other plugins that have ratings, for example, goodreads.

02-24-2017, 06:13 PM	#38
Oddball Member Posts: 11 Karma: 10 Join Date: Nov 2016 Device: Kindle Desktop	The way this plugin operates reminds me of the way the RIAA beats up on their own customers. Amazon is just doing the same thing. And this WILL cost them the same way it cost the RIAA in big money and loss of market share. The MASSIVE change done to amazon ebooks to the kfx and kf8 files formats is just on more of example of a company dong its damnedest to shoot itself in the head (just like Coca-cola with New Coke). I guess bezos has forgotten that under the Sony FAIR USE that a backup of digital media is legal.. all i see this is preventing that. Enough of that, since the 2.80 update I get the Captcha error after ONE metadata download...AMAZON SUCKS.

02-25-2017, 09:55 AM	#39
kovidgoyal creator of calibre Posts: 45,221 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Probably time for me to look into replacing the amazon builtin plugin with some other site. Given the way amazon has been behaving of late, the less things in calibre that depend on it the happier I'll be. Any recommendations from people that regularly use other general purpose metadata sources?

02-25-2017, 12:42 PM	#40
theducks Well trained by Cats Posts: 30,914 Karma: 60358908 Join Date: Aug 2009 Location: The Central Coast of California Device: Kobo Libra2,Kobo Aura2v1, K4NT(Fixed: New Bat.), Galaxy Tab A	I use both Fantastic Fiction sites (I am over 21... by a long shot ) I still get "Not Founds" (mostly for self pub found at Amazon USA), so something else is needed to fll those gaps

02-26-2017, 04:47 AM	#42
kovidgoyal creator of calibre Posts: 45,221 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	There are two fantastic fiction sites? I'm only familiar with one: www.fantasticfiction.com and as far as I know, it focuses only on relatively recent books in genres like SF, Romance, Mystery etc. I dont know how suitable it would be as a general purpose metadata source. Looking at its homepage it says it has about half a million books. I suppose that's not too bad, it's about a quarter of amazon's catalog.

02-26-2017, 01:19 PM	#44
kovidgoyal creator of calibre Posts: 45,221 Karma: 27110894 Join Date: Oct 2006 Location: Mumbai, India Device: Various	How does one get to the adult version? I can see no mention of it on the FF home page and googling it returns nothing useful.

Advert

Advert