Calibre: download metada result in hung worker

[BetterRed]

Originally Posted by Sabardeyn

No, when I don't press them, they don't work for me!

Don't think I ever knew you could delete the cover from the Book Details though.

Thanks for letting me know.

Thanks for raising the issue, I didn't know until I looked for the Remove in Book Details that one can right click copy/paste a cover from one book to another via Book Details - for me that's quite handy.

BR

[Maverynthia]

Originally Posted by Sabardeyn

Yes, thank you, that was the discussion I was trying to reference but could not re-located. While it isn't applicable here, it does seem to have helped us find the culprit.


Maverynthia,
Huh that is kind of odd because I was just using the Search the Internet plugin to look up a few covers yesterday. That plugin searched Google for images without issue so the two plugins must submit different requests to Google. I have not previously used Google Image Search.

The Google Search works fine, it's just the Image Search for covers using the "Download Metadata/Covers", it does fail on a single request as well, BUT it just brings up no covers and gives an error "No valid covers found" with the red X and then I just select OK with the current default cover selected and go back to the screen. I suppose it could be failing when there is no default cover and it can't find a cover either. Like the cover is just that brown book icon.

EDIT: I updated to version 1.5.0 and from another thread I had on here I was going to Advance > Misc. and saw the "Number of jobs 3" and the "Max jobs to cores". Since i have a dual core I put number of jobs to 2. Now it's giving me a DIFFERENT error! I'm going to try with ONE job and see what happens.

calibre, version 1.5.0 (win32, isfrozen: True)
Failed to download metadata: Failed: Download metadata for 137 books

Starting job: Download metadata for 137 books
Job: "Download metadata for 137 books" failed with error:
Traceback (most recent call last):
File "site-packages\calibre\gui2\threaded_jobs.py", line 83, in start_work
File "site-packages\calibre\gui2\metadata\bulk_download.py", line 236, in download
File "site-packages\calibre\utils\ipc\simple_worker.py", line 160, in fork_job
File "site-packages\calibre\utils\ipc\simple_worker.py", line 81, in communicate
WorkerError: Something strange happened. The worker process was aborted without an exception.

Called with args: ([808, 807, 802, 800, 801, 799, 798, 795, 796, 797, 794, 793, 791, 792, 789, 790, 787, 788, 786, 784, 785, 783, 782, 781, 780, 779, 778, 777, 776, 774, 775, 773, 772, 771, 769, 770, 767, 768, 766, 765, 764, 763, 761, 762, 759, 758, 760, 756, 757, 755, 753, 754, 751, 752, 750, 749, 748, 747, 746, 744, 745, 743, 742, 740, 741, 739, 738, 737, 736, 735, 733, 734, 732, 730, 731, 729, 728, 727, 726, 725, 724, 723, 722, 721, 720, 719, 718, 717, 715, 716, 714, 712, 713, 710, 711, 709, 708, 707, 706, 705, 703, 704, 702, 701, 700, 699, 697, 698, 695, 696, 694, 693, 692, 691, 690, 688, 689, 687, 686, 685, 684, 683, 682, 681, 680, 679, 678, 677, 676, 675, 674, 673, 672, 671, 670, 668, 669], u'S:\\TEMP\\calibre_ftto8o\\knicv2_metadata_bulk.l og', <calibre.db.legacy.LibraryDatabase object at 0x0460A390>, False, True, None) {u'notifications': <Queue.Queue instance at 0x08FEC828>, u'abort': <threading._Event object at 0x0900CBB0>, u'log': <calibre.utils.logging.GUILog object at 0x0900CA70>}

[Sabardeyn]

Kovid,
As an alternate idea, could a sandbox and/or Window's UAC cause the same type of error response?

I'm kind of thinking that maybe while Maverynthia was busy a dialog asking for permission was either denied or completely missed. The lack of approval back then is being remembered, and the UAC/sandbox is quietly continuing to deny the request, without asking again.

Maverynthia,
I just enabled Google Image Search and tried to find a missing book cover for one book. I got responses from each site (Amazon, B&N, Google, Google Image, etc) after waiting a few minutes for the request to be processed. None of them were appropriate, but the request was honored without causing any errors.

My calibre settings are "almost stock" in that I just got this computer 2 months ago and I really haven't customized any settings/preferences/tweaks yet. After spending hours each day editing metadata previously, I got burned out and have been taking a break. So if it's not an antivirus, maybe it is a custom setting?
Hmm... Sorry, kind of grasping at straws , hoping that it reveals something.

[kovidgoyal]

I doubt it, the worker process does not require elevated privileges so UAC is not involved. As I said, the only thing that I know of that can explain the symptoms is an antivirus program.

[BetterRed]

I wonder if its something competing for network resources - P2P, RSS feeds, stock tickers etc - that said I run those sort of things all day, whilst I don't use Calibres metadata & cover downloads much I've not had errors of this sort.

I'd try running Calibre on as close to a bare bones system as is possible.

BR

[Maverynthia]

Originally Posted by kovidgoyal

I doubt it, the worker process does not require elevated privileges so UAC is not involved. As I said, the only thing that I know of that can explain the symptoms is an antivirus program.

Also I'm on Windows XP (SP3) so there is no UAC.
As I said there is NO antivirus program.
I also stated it was ONLY Google Images and only if it can't fine ANY covers, so the problem lies there somewhere.

I wonder if its something competing for network resources - P2P, RSS feeds, stock tickers etc - that said I run those sort of things all day, whilst I don't use Calibres metadata & cover downloads much I've not had errors of this sort.

I'd try running Calibre on as close to a bare bones system as is possible.

BR

I've also said I've run this before with some Amazon MOBIs and the Google Images plugin was just fine. It's choking on some EPUBS with no real internet presence, like those from Smashwords or Kobo.

[DoctorOhh]

Originally Posted by Maverynthia

It's choking on some EPUBS with no real internet presence, like those from Smashwords or Kobo.

First do you still experience this problem? Don't keep the books a secret, let us know so others can try and replicate your problem.

I use Google images and can not replicate your results.

[theducks]

I use the Search the Internet: Google for images all the time: (I look for the OLD, original covers that my paper editions have).
That feature uses the system Browser.

Maybe your system default browser is not the one you use AND it may be opening on another Monitor (that is now OFF or missing) and is waiting for user input.

[Maverynthia]

Originally Posted by DoctorOhh

First do you still experience this problem? Don't keep the books a secret, let us know so others can try and replicate your problem.

I use Google images and can not replicate your results.

I do still experience it, the problem is that when I run a batch it never tells me which book it's failing on so I can't pinpoint it to a specific book. Like it'd be helpful it it says "Failed on <bookname>" or even <filename> so I can go "OH yeah THAT book." i can only see what books succeeded and try to guesstimate around what book it was.

I use the Search the Internet: Google for images all the time: (I look for the OLD, original covers that my paper editions have).
That feature uses the system Browser.

Maybe your system default browser is not the one you use AND it may be opening on another Monitor (that is now OFF or missing) and is waiting for user input.

Uhm? I think you have the wrong thread, this is about the Google Image plugin for Calibre failing on a "Download Metadata/Covers" that you can get when selecting books and right-clicking.

[Sabardeyn]

Maverynthia,
Theducks simply offered an alternate way to try and find cover images. You will find that their are generally a couple of ways in calibre to complete any task. "Search the Internet" is the name of a plugin, while Google Images is one of the optional searches the plugin can perform.


Anyway, since you're still encountering the problem, why not try a smallish bunch of books and if you get an error, try again for each book individually? You can find which books need a cover by performing a cover:false search, then highlight some books and perform your standard metadata download. If any of the books fail, then go through those books one by one, to find the problem book (one of those without a cover) by repeating the search individually.

(Just so you know, you might find that some of those dozen books are missing. If some of the book cover's are found, those books can be removed by calibre because the search criteria no longer applies. That is, covers were found for those books so cover:false no longer applies.)

Once you've found one of the problem books, let us know the name of it and where you got the ebook file from (if you remember).

[Maverynthia]

Originally Posted by Sabardeyn

Maverynthia,
Theducks simply offered an alternate way to try and find cover images. You will find that their are generally a couple of ways in calibre to complete any task. "Search the Internet" is the name of a plugin, while Google Images is one of the optional searches the plugin can perform.

I didn't know that was the name of a plugin. (Which is why I got confused about missing monitors and stuff). I was using the built-in Google Images one. I'll look into that plugin in the meantime.

Anyway, since you're still encountering the problem, why not try a smallish bunch of books and if you get an error, try again for each book individually? You can find which books need a cover by performing a cover:false search, then highlight some books and perform your standard metadata download. If any of the books fail, then go through those books one by one, to find the problem book (one of those without a cover) by repeating the search individually.

(Just so you know, you might find that some of those dozen books are missing. If some of the book cover's are found, those books can be removed by calibre because the search criteria no longer applies. That is, covers were found for those books so cover:false no longer applies.)

Once you've found one of the problem books, let us know the name of it and where you got the ebook file from (if you remember).

I'll go do that. I tired it before on a group of 4 books, yet on an individual search they behaved themselves, only to redo them on a multiple search and have them go 'nope'. I'll keep mucking with it to try and narrow it down to one or a couple books.
(Just frustrating when my group of Amazon MOBIs breeze through fine, yet these EPUBs are all 'nope, covers are for losers!')

I should also state that my Amazon Free is a different library than my Main Library. They are both under /My Documents however.

OK how exactly do I do the cover:false search? (Where am I putting that? and does it respect the R-click > Download metadata and covers?)

Where are the books from? Kobo. I took them from my Kobo directory, shoved them into Calibre then converted them from EPUB into EPUB (some of the books claim to also have been on Smashwords at some point in their life but all were from Kobo). There were no errors with conversion.

EDIT: I narrowed it down to "The Three Musketeers by Alexandre Dumas" it's the Kobo version that has been reconverted into EPUB by Calibre. I tried renaming a DIFFERENT book to the title and author and left the rest blank and it still hangs. I even tried editing the Author and still nothing.
(There are others that cause it to hang, but this is ONE of them)

[theducks]

Metadata Search will use listed identifiers when they exist

If you changed just the title,
the search is still looking by for data by identifier (and if you have those allowed in the source configuration, will correct them as part of the download).


Now you have found the culprit,
Try a test: Clear all the identifiers for that book and see if the search hangs

[Maverynthia]

Originally Posted by theducks

Metadata Search will use listed identifiers when they exist

If you changed just the title,
the search is still looking by for data by identifier (and if you have those allowed in the source configuration, will correct them as part of the download).


Now you have found the culprit,
Try a test: Clear all the identifiers for that book and see if the search hangs

I changed the title, author and removed all other information. So Google Images only had the title and author to go off of, which is all that's available on the REAL "Three Musketeers" book. Both books failed.

If the problem is that Google Images NEEDS and identifier and hangs when none is available, that's a problem for books without ids.

[Sabardeyn]

Originally Posted by Maverynthia

OK how exactly do I do the cover:false search? (Where am I putting that? and does it respect the R-click > Download metadata and covers?)

1. Click in the Search field, above the book view/list.
2. Type the following expression exactly as shown: cover:false
3. Press Enter

Your book view will be updated with a subset of your books. Only books that match the Search criteria will be shown. In this case we're looking for a field/reserved keyword called cover and want to know if it's true (has a value) or false (has no value). So, basically, that search is just looking to see if calibre's got a separate cover image file associated with the ebook. (The tail end of that last sentence simply means calibre is not looking inside of the ebook, just on the hard drive. This is important to note because the book may already have a cover inside.)

Originally Posted by Maverynthia

Where are the books from? Kobo. I took them from my Kobo directory, shoved them into Calibre then converted them from EPUB into EPUB (some of the books claim to also have been on Smashwords at some point in their life but all were from Kobo). There were no errors with conversion.

Hmm... I wonder if any of the problem books originally had some kind of Digital Rights Management (DRM - its a fancy term for copy protection) and failed to convert correctly? Sometimes calibre will appear to convert a book properly, without errors, but when trying to read the book, there are blank pages, incorrectly formatted text, and is otherwise just a mess. This is generally due to the vagaries of ebook construction, not calibre doing anything wrong.

If you still have, or can re-download, the original Kobo ebook of the Three Musketeers, try and see if you can open it through calibre's viewer. Or any other ereader software that is supposed to handle EPUB format. Does the book open and display correctly? Do you encounter any problems when paging through the book? Tell us what ereader you used and what the results were.

[Maverynthia]

Originally Posted by Sabardeyn

1. Click in the Search field, above the book view/list.
2. Type the following expression exactly as shown: cover:false
3. Press Enter

Your book view will be updated with a subset of your books. Only books that match the Search criteria will be shown. In this case we're looking for a field/reserved keyword called cover and want to know if it's true (has a value) or false (has no value). So, basically, that search is just looking to see if calibre's got a separate cover image file associated with the ebook. (The tail end of that last sentence simply means calibre is not looking inside of the ebook, just on the hard drive. This is important to note because the book may already have a cover inside.)

Ah OK, though the book already had a cover, just not a very good one. I think on conversion it took the EPUB cover.

Hmm... I wonder if any of the problem books originally had some kind of Digital Rights Management (DRM - its a fancy term for copy protection) and failed to convert correctly? Sometimes calibre will appear to convert a book properly, without errors, but when trying to read the book, there are blank pages, incorrectly formatted text, and is otherwise just a mess. This is generally due to the vagaries of ebook construction, not calibre doing anything wrong.

If you still have, or can re-download, the original Kobo ebook of the Three Musketeers, try and see if you can open it through calibre's viewer. Or any other ereader software that is supposed to handle EPUB format. Does the book open and display correctly? Do you encounter any problems when paging through the book? Tell us what ereader you used and what the results were.

Actually the Digital Rights were stripped when I converted it. It actually wasn't displaying properly until AFTER it was converted. After thus it was fine. But it displaying isn't the problem here. Getting the cover image is.

I tried again with a MOBI I had, it failed again. For some reason the Google Image plugin hates The Three Musketeers.

EDIT: here's the file. It's a KOBO non DRM epub that seems to have come from Gutenberg. But as I said, it's not like the file itself seems to matter in this case. Just the name/author.
https://dl.dropboxusercontent.com/u/...950abd338.epub