Downloading metadata in versions after 0.8.0

[amyk0202]

I'm not sure how to phrase this question. I am updating my library author by author & adding metadata to each book--several thousand. In version 0.8.0, I can take a list of books, many with the same title & author but some problem like the columns being switched, etc., & get the same (correct) results. I've tried updating calibre up to 0.8.5 & I can take a list of books & download metadata & covers & it will give me the correct results for some, but change everything completely for others--new author, new title. This stands out as I am going author by author & all of a sudden I have books that don't belong, but it is inconvenient since I have to open the book up to find out what it really is. This wouldn't be a huge deal as I can just keep using 0.8.0 until I am done with coding my library, but my daughter broke her Nook so I bought her a Pocketbook 360+ & 0.8.0 doesn't recognize it. Now I have to put her memory card in my older Pocketbook to put files on it. It makes me nervous to keep switching chips since I have already ruined one & had to start all over (I have bad luck ). Is there a way to make the newer versions of calibre act the same way as the older version when downloading metadata?

[kovidgoyal]

post the title, author and ids for a book where calibre gets it wrong.

[amyk0202]

Hi Kovid,
I updated to 0.8.6 & got the same result as in the past. The specific books this time are:
"The Bride Hunt" by Margo Maguire, isbn:9780060837143, amazon:0060837144, douban:2660190, ff:m/margo-maguire/bride-hunt-avon-historical-romance, goodreads:551192; and "Temptation of the Warrior" by Margo Maguire, isbn:9780061256370, amazon:0061256374, ff:m/margo-maguire/temptation-of-warrior, goodreads:1851859.

I saved a copy of the log this time & it looks to me like the douban books source is coming up with the wrong information & that is what it ends up with.

This is a portion of the log:

Starting job: Download metadata for 6 books
Running identify query with parameters:
{u'authors': [u'Margo Maguire'], u'identifiers': {}, u'timeout': 30, u'title': u'The Bride Hunt'}
Using plugins: Goodreads, Fantastic Fiction, Amazon.com, ISBNDB, Douban Books
The log from individual plugins is below

****************************** Fantastic Fiction ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.78 Safari/532.5')]
Found 1 results
Downloading from Fantastic Fiction took 1.28099989891

---
Title : The Bride Hunt
Author(s) : Margo Maguire
Publisher : Avon Books
Series : Conqueror #2
Published : 2006-01-01T00:00:00+00:00
Identifiers : isbn:9780739461112, ff:m/margo-maguire/bride-hunt-avon-historical-romance
Comments : I deleted the comments.

Querying: http://db.fantasticfiction.co.uk/cgi...+Margo+Maguire
FantasticFiction url: u'http://www.fantasticfiction.co.uk/m/margo-maguire/bride-hunt-avon-historical-romance.htm'

************************************************** ******************************

****************************** Douban Books ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.2.11) Gecko/20101012 Firefox/3.6.11')]
Found 10 results
Downloading from Douban Books took 6.98099994659


---
Title : Click
Author(s) : David Almond & Eoin Colfer & Roddy Doyle & Deborah Ellis & Nick Hornby & Margo Langan & Gregory Maguire & Ruth Ozeki & Linda Sue Park & Tim Wynne-jones
Publisher : Arthur A. Levine Books
Tags : Fun, NickHornby, Novel, Recommendation, 小说, 欧美
Rating : 0.0
Published : 2007-10-01T17:21:11.582000+00:00
Identifiers : douban:2292240, isbn:9780439411387
Comments : I deleted the comments.

The book "Click" is what I got after it finished updating for both the Margo Maguire books. For now I'll try taking douban books off & see what happens.

[kovidgoyal]

I'd suggest turning off douban in that case. I'll ping the developer of the douban plugin and ask him to take a look.

[lifanxi]

The search function of Douban.com didn't provide the facility to search by specified book title or author, it treat all the information as "search keywords". Besides, Douban.com is optimized for Chinese books only, so for English keywords, it may return not-so-good results.

For the solution, I suggest non-Chinese user disable Douban.com plugin in calibre to avoid such problem. In the long term, I will try to improve the search accuracy in Douban.com by adding extra code in the Douban.com plugin.

Sorry for the inconvenience.

[goldyman]

I'm having a similar issue with versions after 0.8.0. Cannot download any metadata using just Amazon and/or Google. The most recent log is tagged below, though note that I get similar errors with every book I try.

One thought is that I might be having an issue because I must use a proxy server, but a look in Terminal shows that calibre recognizes that fact and has the proxy set correctly.

Here's the log:

calibre, version 0.8.6
ERROR: No matches found: <p>Failed to find any books that match your search. Try making the search <b>less specific</b>. For example, use only the author's last name and a single distinctive word from the title.<p>To see the full log, click Show Details.

Starting download
Query: title:Walden; authors:Henry David Thoreau; isbn:9781613820162
Running identify query with parameters:
{u'authors': [u'Henry David Thoreau'], u'identifiers': {u'isbn': u'9781613820162'}, u'timeout': 30, u'title': u'Walden'}
Using plugins: Google, Amazon.com
The log from individual plugins is below

****************************** Google ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1')]
Found 0 results
Downloading from Google took 180.11074996
Failed to make identify query: u'http://books.google.com/books/feeds/volumes?q=isbn%3A9781613820162&max-results=20&min-viewability=none&start-index=1'
Traceback (most recent call last):
File "site-packages/calibre/ebooks/metadata/sources/google.py", line 296, in identify
File "site-packages/mechanize/_mechanize.py", line 199, in open_novisit
File "site-packages/mechanize/_mechanize.py", line 230, in _mech_open
File "site-packages/mechanize/_opener.py", line 193, in open
File "site-packages/mechanize/_urllib2_fork.py", line 344, in _open
File "site-packages/mechanize/_urllib2_fork.py", line 332, in _call_chain
File "site-packages/mechanize/_urllib2_fork.py", line 1142, in http_open
File "site-packages/mechanize/_urllib2_fork.py", line 1118, in do_open
URLError: <urlopen error timed out>

************************************************** ******************************

****************************** Amazon.com ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.78 Safari/532.5')]
Found 0 results
Downloading from Amazon.com took 30.2416381836
Amazon timed out. Try again later.

************************************************** ******************************
The identify phase took 180.12 seconds
The longest time (180.110750) was taken by: Google
Merging results from different sources and finding earliest publication dates
We have 0 merged results, merging took: 0.00 seconds

[kovidgoyal]

Those are timeouts connecting to the servers. I doubt you'll be able to use any of calibre's network related features, not just downloading metadata.

[goldyman]

Originally Posted by kovidgoyal

Those are timeouts connecting to the servers. I doubt you'll be able to use any of calibre's network related features, not just downloading metadata.

Is there anyway to make sure calibre is using the correct proxy server? I'm not well-versed in Terminal (using Mac OS X 10.6.7) and cannot find server settings in any of calibre's easily accessible prefs.

[kovidgoyal]

I'll add some code that will show the currently used proxies in preferences for the next release.

[goldyman]

Originally Posted by kovidgoyal

I'll add some code that will show the currently used proxies in preferences for the next release.

Awesome! Thanks. Appreciate all you do for calibre.