[Old Thread] Applications hang when converting or adding more than 1 book at a time

[infoquake123]

All applications hang when converting more than 1 book at a time. I have a library with about 15k books. I have no problems when i convert an e-book individually but if i try to convert 20 books or so, all applications including calibre and internet explorer hang. After 2hours , everything returns to normal and the applications no longer hang. If i try to convert 100 or more books, all applications including calibre hang and after 5-6 hours, a message pops up that calibre is no longer working and that i should shut it off. Typically any conversion that is more than 50 books will cause all applications to hang immediately and many hours later eventually a message will pop up saying that calibre is not working. I'm a newbie and not sure what information to provide. Pls let me know I'm tried switching off my antivirus but that does not help. I have no idea what to do. Would really appreciate it if someone could help me. I'm using a 32 bit system with windows 7 home by the way.

[kovidgoyal]

Go to preferences->misc and set the max number of simultaneous jobs to 1 and set the worker priority to low.

[theducks]

Originally Posted by infoquake123

All applications hang when converting more than 1 book at a time. I have a library with about 15k books. I have no problems when i convert an e-book individually but if i try to convert 20 books or so, all applications including calibre and internet explorer hang. After 2hours , everything returns to normal and the applications no longer hang. If i try to convert 100 or more books, all applications including calibre hang and after 5-6 hours, a message pops up that calibre is no longer working and that i should shut it off. Typically any conversion that is more than 50 books will cause all applications to hang immediately and many hours later eventually a message will pop up saying that calibre is not working. I'm a newbie and not sure what information to provide. Pls let me know I'm tried switching off my antivirus but that does not help. I have no idea what to do. Would really appreciate it if someone could help me. I'm using a 32 bit system with windows 7 home by the way.

That's not a 'Hang"
That's your system gasping for spare CPU cycles
Conversions are intensive use of CPU and I/O

Win 7 on a 32 bit machine . IMHO That puppy needs to be maxed-out in RAM. Tame your Anti-virus to Ignore Calibre program Directory AND your Library folder (Note that The A/V normally scans your book before it is imported and the downloaded Calibre program before it is installed. Moving the book into a Library/converting needs no further scans.)

In Calibre; Preferences; Behavior; Set Job priority to LOW, that will let other processes function a bit.

[infoquake123]

Setting the max number of simultaneous jobs to one and seeting the job priority to low seems to have solved the problem. Just tried converting 20 books and there was no hanging or any other problems. And as suggested, I have excluded Calibre from my norton antivirus scan. Thanks a lot guys.

Now that i have that major headache resolved, i have questions about 2 small problems. I wanted to overide the image icons used in Calibre. In the manual it says that i should not change the images stored in the resources sub-folder of the calibre install location as my changes will get overwritten the next time I update calibre. And that i should instead, go to Preferences->Advanced->Miscellaneous and click Open calibre configuration directory, create a sub-folder called resources and place the files i want to override in it.

When I went to the open calibre configuration directory, there was already a sub-folder called resources and another folder called images within that. And there was another sub-folder called generate_images within the images folder. As instructed, I placed my images ( that had the same names as the images in the images stored in the resources sub-folder of the calibre install location) , inside the image folder but it did not change the image icons in calibre to mine even after i restarted calibre. Why is this ?

Also, since that did not work, I tried replacing some of the default images in the resources sub-folder of the calibre install location with my images that had the same name . ( I know that the images will be overridden back to the default when i update calibre in future ) That worked for some image icons like library.png and book.png but when i replaced the catalog.png with my own catalog.png, the image icon for the catalog disappeared. Why is this ? Does the pixel size have to be the same or something ?

Also is there any way to change the roman numerals used to show the series info when generating the covers to normal numbers ?

P.S On a separate note, Calibre is an amazing program and i seem to be able do more and more with it as i learn more about it. Kovid, just wanted to let u know that u have done an amazing job.

[theducks]

Originally Posted by infoquake123

Now that i have that major headache resolved, i have questions about 2 small problems. I wanted to overide the image icons used in Calibre. In the manual it says that i should not change the images stored in the resources sub-folder of the calibre install location as my changes will get overwritten the next time I update calibre. And that i should instead, go to Preferences->Advanced->Miscellaneous and click Open calibre configuration directory, create a sub-folder called resources and place the files i want to override in it.
When I went to the open calibre configuration directory, there was already a sub-folder called resources and another folder called images within that.

Follow the instruction Exactly as to LOCATION. If the folder already exists...Use it, don't try and drill down deeper

Also is there any way to change the roman numerals used to show the series info when generating the covers to normal numbers ?

Preferences; Look and Feel; Book Details tab; Top Right

[sheygetz]

Originally Posted by theducks

In Calibre; Preferences; Behavior; Set Job priority to LOW, that will let other processes function a bit.

If I may take this up ...
I would actually expect to have more than "a bit" if the prio is set to low. But calibre-parallel uses 50% of CPU anyway (XP, double core). More precisely I would expect, that on low I could use my computer pretty much normally. So maybe we should then have a "very low" setting? Personally, I would not mind if a conversion (typically epub>mobi for me) took 10 minutes. But to hardly be able to even surf this forum while it converts is simply not helpful.

Hendrik

[theducks]

Originally Posted by sheygetz

If I make take this up ...
I would actually expect to have more than "a bit" if the prio is set to low. But calibre-parallel uses 50% of CPU anyway (XP, double core). More precisely I would expect, that on low I could use my computer pretty much normally. So maybe we should then have a "very low" setting? Personally, I would not mind if a conversion (typically epub>mobi for me) took 10 minutes. But to hardly be able to even surf this forum while it converts is simply not helpful.

Hendrik

Priority and 50% CPU is not the same.
Priority is who gets to use the CPU (first), not how much.
BOINC uses both (Hyperthread on P4) cores at low CPU priority.

If your data bus is clogged by poor (disk,graphics....) I/O, priority won't help much.
Most common is A/V wedged into the chain (where it is not needed), but any file monitoring (auto replication? ) will suck performance.

On this Dell, the Graphics and Disk (the whole PCI bus) seem to be on the same interrupt and the BIOS has no provision to assign INT by slots other mainboards either split slot INT's (so I just move boards) or allow direct assignment
I surf and convert all the time. Streaming and converting can be hit or miss, so I only run 1 convert job at a time if I want to watch a quality stream.
Find your bottleneck
(then fix it )

[PeterT]

You might like to try a program called Process Lassoo and see if it's automatic adjustments of program priorities might make your PC more responsive during conversions.

Did you also limit the number of jobs calibre can run?

[DoctorOhh]

Originally Posted by sheygetz

If I make take this up ...

You did take it up, but as a general rule reactivating a thread that has been dead this long is frowned upon.

Originally Posted by sheygetz

I would actually expect to have more than "a bit" if the prio is set to low. But calibre-parallel uses 50% of CPU anyway (XP, double core). More precisely I would expect, that on low I could use my computer pretty much normally. So maybe we should then have a "very low" setting? Personally, I would not mind if a conversion (typically epub>mobi for me) took 10 minutes. But to hardly be able to even surf this forum while it converts is simply not helpful.

I don't run on a dual core XP machine anymore but when I did setting Job priority to low did allow me to surf fine while converting. When I view the running processes on my Windows 7 machine there are no processes that are not normal or high. Additionally in Windows 7 you have the below choices to set as job priorities, note that low is the bottom of the list. There is no very low.

Realtime
High
Above Normal
Normal
Below Normal
Low

You might want to ensure that calibre library is exempt from virus or malware screening.

[BetterRed]

@sheygetz - Since I followed KG's advice and set

Behaviour->Job Priority from Normal to Low;
Miscellaneous->Max. simultaneous... jobs from 3 to 1 and
Miscellaneous->Limit the max. simultaneous ... cores from checked to unchecked (my addition).

the calibre-parallel process doing the conversion rarely gets above 10% of one core. And on my rig doing bulk conversions serially gives better throughput than doing them in parallel.

BR

[sheygetz]

(That should've read "may take" ...)

Thanks for taking the time to reply everyone.
@theducks: That sounds very knowledgable, but as I am unfortunately not I only have a vague idea what you're saying. I take it, there's nothing much I can do, if I'm not into messing with boards, interrupts (remember those from WfW 3.11), slots, bios,etc. Incidentally I am on a Dell (6+ years running).

@ DoctorOhh: "There is no very low" Yeah, well that was just a way of phrasing that Calibre's low is obviously not low enough, at least for Dell & me. I will fine tune Avast to exempt the said directories. Ta.

@Peter and BetterRed: Yes, I did limit the number of parallel jobs to 1. I'll try the core setting too.

Hendrik