Register Guidelines E-Books Search Today's Posts Mark Forums Read

Go Back   MobileRead Forums > E-Book Software > Calibre

Notices

Reply
 
Thread Tools Search this Thread
Old 09-24-2013, 09:27 AM   #1
Maverynthia
Enthusiast
Maverynthia began at the beginning.
 
Maverynthia's Avatar
 
Posts: 30
Karma: 10
Join Date: May 2011
Device: Samsung Galaxy 3 8.0 / LG Optimus F3 / PC
Exclamation Calibre: download metada result in hung worker

I was getting some metadata and though it was because i had 136 books queued, but I took it down to 3 and it still got hung. It still works for an individual book and it would sometimes work for other clusters of books. (I was getting both covers and metadata.)

Starting job: Download metadata for 3 books
Job: "Download metadata for 3 books" failed with error:
Traceback (most recent call last):
File "site-packages\calibre\gui2\threaded_jobs.py", line 83, in start_work
File "site-packages\calibre\gui2\metadata\bulk_download.py", line 236, in download
File "site-packages\calibre\utils\ipc\simple_worker.py", line 160, in fork_job
File "site-packages\calibre\utils\ipc\simple_worker.py", line 68, in communicate
WorkerError: Worker appears to have hung

Called with args: ([786, 782, 783], u'S:\\TEMP\\calibre_qrk2sj\\jc_edd_metadata_bulk.l og', <calibre.db.legacy.LibraryDatabase object at 0x0483D5F0>, True, True, None) {u'notifications': <Queue.Queue instance at 0x08FCA850>, u'abort': <threading._Event object at 0x08EF8C30>, u'log': <calibre.utils.logging.GUILog object at 0x08EF8110>}
Maverynthia is offline   Reply With Quote
Old 09-24-2013, 11:41 AM   #2
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,622
Karma: 4998489
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Try disabling your antivirus, antivirus programs often think that a program that is connecting to lots of websites, like the calibre download worker is malicious
kovidgoyal is offline   Reply With Quote
 
Enthusiast
Old 09-24-2013, 12:27 PM   #3
Maverynthia
Enthusiast
Maverynthia began at the beginning.
 
Maverynthia's Avatar
 
Posts: 30
Karma: 10
Join Date: May 2011
Device: Samsung Galaxy 3 8.0 / LG Optimus F3 / PC
Don't have any antivirus. So it can't be that. It was working yesterday on all the MOBIs I was doing from an Amazon back up. These are all EPUBs.
Maverynthia is offline   Reply With Quote
Old 09-24-2013, 12:28 PM   #4
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,622
Karma: 4998489
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
Well something on your computer is causing the process to hang, if it isn't the antivirus, then it is likely some other security software.
kovidgoyal is offline   Reply With Quote
Old 09-25-2013, 01:22 AM   #5
Maverynthia
Enthusiast
Maverynthia began at the beginning.
 
Maverynthia's Avatar
 
Posts: 30
Karma: 10
Join Date: May 2011
Device: Samsung Galaxy 3 8.0 / LG Optimus F3 / PC
I have no other security software on my computer mainly BECAUSE it's so intrusive and doesn't allow things to work right.
I found further information, it seems to have hung after 32% and I am guessing it was on this book:


****************************** Amazon.com Covers ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (Windows NT 6.2; rv:9.0.1) Gecko/20100101 Firefox/9.0.1')]
Failed to download valid cover
Took 2.6400001049 seconds
No cached cover found, running identify
Failed to make details query: '/Dark-Earth-Jason-Halstead/dp/1451572964/ref=sr_1_1?ie=UTF8&qid=1380082285&sr=8-1'
Traceback (most recent call last):
File "site-packages\calibre\ebooks\metadata\sources\amazon.py ", line 226, in get_details
File "site-packages\mechanize-0.2.4-py2.7.egg\mechanize\_mechanize.py", line 199, in open_novisit
File "site-packages\mechanize-0.2.4-py2.7.egg\mechanize\_mechanize.py", line 216, in _mech_open
BrowserStateError: can't fetch relative reference: not viewing any document

No cover found

************************************************** ******************************

****************************** Open Library Covers ******************************
Request extra headers: [('User-agent', 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)')]
Failed to download valid cover
Took 0.483999967575 seconds
No cover for ISBN: '9781451572964' found

************************************************** ******************************

It seems to hang when it can find no "valid" covers. I processed it manually and it got the metadata fine, however the ONLY covers it could find were "Google Image Search" covers. No others, possibly causing the open library and/or Amazon cover search to shrug and go "what do I do now?"

I also ran a Metadata only and it processed correctly out at 10 minutes:
The identify phase took 8.72 seconds
The longest time (8.485000) was taken by: Google
Merging results from different sources and finding earliest publication dates from the worldcat.org service
We have 9 merged results, merging took: 2.48 seconds


So it's the covers that are causing Calibre to hang. I want to say it's Amazon but I shut it off and it still hangs.
Maverynthia is offline   Reply With Quote
Old 09-25-2013, 06:45 AM   #6
Sabardeyn
Guru
Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.
 
Sabardeyn's Avatar
 
Posts: 629
Karma: 1242364
Join Date: May 2009
Location: The Right Coast
Device: PC (Calibre), Nexus 7 2013 (Moon+ Pro), HTC HD2/Leo (Freda)
Bit of a wild guess here...

Cannot recall which topic it was exactly ATM, but in one of the plugin topics (I think) in the last couple of days, there was a discussion that came down to Amazon having changed their website, thus disrupting the plugin. Perhaps the change has also interfered with calibre's built-in functionality?

(Of course, now that I'm trying to find the topic, I cannot locate it again.)
Sabardeyn is offline   Reply With Quote
Old 09-25-2013, 08:02 AM   #7
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 3,021
Karma: 1682890
Join Date: Mar 2012
Location: NSW Australia
Device: none
Quote:
Originally Posted by Sabardeyn View Post
Bit of a wild guess here...

Cannot recall which topic it was exactly ATM, but in one of the plugin topics (I think) in the last couple of days, there was a discussion that came down to Amazon having changed their website, thus disrupting the plugin. Perhaps the change has also interfered with calibre's built-in functionality?

(Of course, now that I'm trying to find the topic, I cannot locate it again.)
this one maybe - http://www.mobileread.com/forums/sho...ghlight=amazon

BR
BetterRed is offline   Reply With Quote
Old 09-25-2013, 10:45 AM   #8
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,437
Karma: 5560777
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
Quote:
Originally Posted by Sabardeyn View Post
Bit of a wild guess here...

Cannot recall which topic it was exactly ATM, but in one of the plugin topics (I think) in the last couple of days, there was a discussion that came down to Amazon having changed their website, thus disrupting the plugin. Perhaps the change has also interfered with calibre's built-in functionality?

(Of course, now that I'm trying to find the topic, I cannot locate it again.)
The (expired) timeout should have returned a not found.
That is if a reasonable timeout value was still used (? Preferences: Behavior: default network timeout: 5 seconds)
theducks is offline   Reply With Quote
Old 09-25-2013, 12:35 PM   #9
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,622
Karma: 4998489
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
The only way a hang can happen is if some metadata download plugin is hanging and no other plugin is finding covers. I suggest you disable/uninstall any third party metadata download plugins you have, as far I recall, all the builtin plugins use timeouts on network requests, so they cannot hang indefinitely, cant speak for third party plugins.

And update your calibre install to be able to get metadata from amazon again, as pointed out earlier in this thread.
kovidgoyal is offline   Reply With Quote
Old 09-25-2013, 01:39 PM   #10
Maverynthia
Enthusiast
Maverynthia began at the beginning.
 
Maverynthia's Avatar
 
Posts: 30
Karma: 10
Join Date: May 2011
Device: Samsung Galaxy 3 8.0 / LG Optimus F3 / PC
I found out the rouge plugin. It was Google Image search. It seems to now toss up EOFs:

calibre, version 1.4.0
ERROR: Download failed: Failed to download any covers, click "Show details" for details.

Traceback (most recent call last):
File "site-packages\calibre\utils\ipc\simple_worker.py", line 50, in run
File "site-packages\calibre\utils\ipc\__init__.py", line 19, in eintr_retry_call
EOFError


I disabled the Google Image plugin and all the other ones (Amazon, B&N, Goodreads, etc.) went fine. No problems.
I enabled ONLY the Google Image plugin and got the same EOF/hung response. I even took the returned images down from 5 to 3 to 1 and it still all hung up. Which sucks since Google Images is the one that returns the nice ~1000+ res covers.
I have a feeling it wasn't hanging before on the MOBIs as they were all from Amazon and thus Amazon would at least toss a cover out. On these ePubs, some of them had no metadata OR relevant/valid covers at all other than what the EPUB provided and thus Google Image search was left by it's lonesome or something shrugging it's shoulders looking around for an adult to solve the problem.

Google Image even choked on the single metadata "Download Cover", but it just tossed up and error and no images other than the current were provided.
Maverynthia is offline   Reply With Quote
Old 09-25-2013, 03:50 PM   #11
Sabardeyn
Guru
Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.
 
Sabardeyn's Avatar
 
Posts: 629
Karma: 1242364
Join Date: May 2009
Location: The Right Coast
Device: PC (Calibre), Nexus 7 2013 (Moon+ Pro), HTC HD2/Leo (Freda)
Quote:
Originally Posted by BetterRed View Post
Yes, thank you, that was the discussion I was trying to reference but could not re-located. While it isn't applicable here, it does seem to have helped us find the culprit.


Maverynthia,
Huh that is kind of odd because I was just using the Search the Internet plugin to look up a few covers yesterday. That plugin searched Google for images without issue so the two plugins must submit different requests to Google. I have not previously used Google Image Search.


Theducks,
The default calibre values listed under Metadata Download are:
Max time to wait after first match is found: 30
Max time to wait after first cover is found: 60
Which exceed your 5 sec suggestion. So unless the timing under Behaviour has been altered (decreased) they should suffice.


Kovid,
I just (re-)noticed that once you set a cover there is no way to delete it, returning to a null state. Can a "Delete Cover" function be added to the context menu of the cover image portion of the Edit Metadata dialog? Or is there some other method to return to a cover=null state?

I know on a few occasions I've discovered I've made an error with the cover image (different ISBN's cover usually) and I would prefer to have no image rather than a wrong image. Searching for cover:false is possible, cover:inappropriate is not.

Last edited by Sabardeyn; 09-25-2013 at 03:53 PM. Reason: Clarify comment.
Sabardeyn is offline   Reply With Quote
Old 09-25-2013, 04:25 PM   #12
BetterRed
null operator
BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.BetterRed ought to be getting tired of karma fortunes by now.
 
Posts: 3,021
Karma: 1682890
Join Date: Mar 2012
Location: NSW Australia
Device: none
Quote:
Originally Posted by Sabardeyn View Post
Can a "Delete Cover" function be added to the context menu of the cover image portion of the Edit Metadata dialog? Or is there some other method to return to a cover=null state?
These don't work for you ?

BR
Attached Thumbnails
Click image for larger version

Name:	Capture.JPG
Views:	84
Size:	123.5 KB
ID:	112139  
BetterRed is offline   Reply With Quote
Old 09-25-2013, 05:01 PM   #13
theducks
Grand Sorcerer
theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.theducks ought to be getting tired of karma fortunes by now.
 
theducks's Avatar
 
Posts: 14,437
Karma: 5560777
Join Date: Aug 2009
Location: The (original) Silicon Valley, USA
Device: Galaxy Tab 2, Astak Pocket Pro, K4NT
@Sabardeyn
I was referring to the overall network setting

I did not notice the ones you mentioned
theducks is offline   Reply With Quote
Old 09-25-2013, 05:25 PM   #14
Sabardeyn
Guru
Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.Sabardeyn ought to be getting tired of karma fortunes by now.
 
Sabardeyn's Avatar
 
Posts: 629
Karma: 1242364
Join Date: May 2009
Location: The Right Coast
Device: PC (Calibre), Nexus 7 2013 (Moon+ Pro), HTC HD2/Leo (Freda)
Quote:
Originally Posted by BetterRed View Post
These don't work for you ?
No, when I don't press them, they don't work for me!

I've been mentally screening that entire group of buttons out from the Edit Metadata dialog. I think that goes back to when Download Metadata was first introduced; getting bombarded with all those (frequently inappropriate) covers, tags, etc just made me blank the buttons out. Too much extra editing and re-editing required!

Don't think I ever knew you could delete the cover from the Book Details though.

Thanks for letting me know.
Sabardeyn is offline   Reply With Quote
Old 09-25-2013, 11:11 PM   #15
kovidgoyal
creator of calibre
kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.kovidgoyal ought to be getting tired of karma fortunes by now.
 
kovidgoyal's Avatar
 
Posts: 25,622
Karma: 4998489
Join Date: Oct 2006
Location: Mumbai, India
Device: Various
@Maverynthia: Google Images works fine for me. It is the most heavyweight plugin, since it uses a complete browser instance, which runs in a separate worker process, the error that you posted means the worker process is not responding. The only way I know of for that to happen is an antivirus to block the process for suspicious activity, other than that, I'm out of ideas. THe code itself times out if google does not respond: https://github.com/kovidgoyal/calibr..._images.py#L95
kovidgoyal is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Calibre keeps getting hung... kingtermite Library Management 3 02-22-2012 03:31 PM
[issue] attempting to download covers can result in exception asxpwpeukrun Calibre 2 07-01-2010 05:54 PM
Calibre worker processes nickredding Calibre 1 04-24-2010 01:01 PM
Collection metada johansolo ePub 2 08-22-2009 08:32 PM
Calibre 0.5.14 on Mac - Weird Send to PRS-505 Result? danviento Calibre 0 07-16-2009 06:28 PM


All times are GMT -4. The time now is 02:13 PM.


MobileRead.com is a privately owned, operated and funded community.