Calibre Freezing

[Rhay]

First let me say, I have my library on an external HD. I know that isn't ideal. However, my problem is calibre will freeze when it seems to hit a book it can't add, hangs for 15 min-12 hours then will suddenly (if i'm lucky) start moving forward again. There are folders of books that I have no problem adding (~400 files, including cover and meta and done in 4 minutes) then others that freeze for longer periods (300 files). Is there a setting somewhere that determines the wait-time for adding books?

[theducks]

Originally Posted by Rhay

First let me say, I have my library on an external HD. I know that isn't ideal. However, my problem is calibre will freeze when it seems to hit a book it can't add, hangs for 15 min-12 hours then will suddenly (if i'm lucky) start moving forward again. There are folders of books that I have no problem adding (~400 files, including cover and meta and done in 4 minutes) then others that freeze for longer periods (300 files). Is there a setting somewhere that determines the wait-time for adding books?

As long as that drive is not a Networked drive, it should be fine. USB, ESATA,or any of the MAC solutions are good as long as you stay within the distance limits.

[Rhay]

No, its not networked. Its plugged directly into my system. I'm currently adding 373 Books, and calibre has been stuck on 21% for ~30 minutes.

and just after hitting post - it started moving again.

[Rhay]

Here is an example of the error I get when it freezes like this -

calibre, version 3.48.0
WARNING: Errors while adding: There were problems adding some files, click "Show details" for more information


----------------------------------------------------------------------
Failed to add the file D:\Books\Books\H\Hand book of pharmacy and thera - Eli Lilly.pdf to the book: Hand book of pharmacy and thera
With error:
Traceback (most recent call last):
File "site-packages\calibre\gui2\add.py", line 445, in add_formats
File "site-packages\calibre\db\cache.py", line 1435, in add_format
File "site-packages\calibre\db\cache.py", line 1400, in _do_add_format
File "site-packages\calibre\db\backend.py", line 1552, in add_format
File "shutil.py", line 63, in copyfileobj
IOError: [Errno 13] Permission denied

Is this talking about file ownership security settings in regards to permission?

[BetterRed]

My libraries are on external drives, never had a problem with them - they used to be on USB 2 ports they're now on USB 3 ports.

When I was adding large quantities I found I got more done by setting Preferences->Miscellaneous->Max. simultaneous conversion/news download/plugin jobs: to 1 and not checking Limit the max. simultaneous jobs to the available CPU cores

And don't do any automatic conversions when you add - that setting is in Add Books->Adding actions.

BR

[Rhay]

I coulda swore I read somewhere that having it on an Ext HD was a no no...I was half convinced that was the problem...

I just started another batch cause I just did 2 smaller ones with no problems, and 5% into this one hit another snag. I'll have to give your settings a try. Thanks for the suggestion!

[BetterRed]

Originally Posted by Rhay

Here is an example of the error I get when it freezes like this -

calibre, version 3.48.0
WARNING: Errors while adding: There were problems adding some files, click "Show details" for more information


----------------------------------------------------------------------
Failed to add the file D:\Books\Books\H\Hand book of pharmacy and thera - Eli Lilly.pdf to the book: Hand book of pharmacy and thera
With error:
Traceback (most recent call last):
File "site-packages\calibre\gui2\add.py", line 445, in add_formats
File "site-packages\calibre\db\cache.py", line 1435, in add_format
File "site-packages\calibre\db\cache.py", line 1400, in _do_add_format
File "site-packages\calibre\db\backend.py", line 1552, in add_format
File "shutil.py", line 63, in copyfileobj
IOError: [Errno 13] Permission denied

Is this talking about file ownership security settings in regards to permission?

It could be that Windows has put a security block on that PDF (and others), I can't remember the term that's used but if you look at the File Properties there's a button to get rid of it.

BR

[Rhay]

If that's the case, and there is some way to weed these types of files out BEFORE attempting an add, that would be great. Otherwise when it hangs up, a serious one (like over night), it tends not to abort, and requires a reboot. The hang, and the reboot add significant time getting this done. If we can cut out the middle man somehow that would be ideal.

[BetterRed]

Originally Posted by Rhay

I coulda swore I read somewhere that having it on an Ext HD was a no no...I was half convinced that was the problem...

I just started another batch cause I just did 2 smaller ones with no problems, and 5% into this one hit another snag. I'll have to give your settings a try. Thanks for the suggestion!

Network storage can be a problem, but any current local storage should be OK.

BR

[BetterRed]

Originally Posted by Rhay

If that's the case, and there is some way to weed these types of files out BEFORE attempting an add, that would be great. Otherwise when it hangs up, a serious one (like over night), it tends not to abort, and requires a reboot. The hang, and the reboot add significant time getting this done. If we can cut out the middle man somehow that would be ideal.

This is 'thing' I was thinking of ==>> How to: Unblock a downloaded file to avoid security warnings – Delay's Blog

To get rid of the "Unblocks" en-masse try this ==>> Easily Unblock All Files in a Directory Using PowerShell | Scripting Blog

And don't convert as you add, a single PDF->EPUB conversion can take 24 hours or more.

BR

[Rhay]

Yea, I don't do conversions either. Fantastic, thank you. I'll give this a try and get back to you.