Slow -- excessive cpu utilization

[Ozzy]

Hey All,

I'm new to the forums and to calibre as well, having just installed it yesterday. Since then, i've been having trouble with excessive cpu utilization every time I use the application.

For instance, importing arouind one hundred files, (including pdfs, docs, rtf and lits) took over an hour and the cpu usage was at max during this entire time.

Also if i just click from one title to another it can utilize anywhere from 60-90% of my cpu, scrolling does the same.

Is it normal that it is this sluggish? and a cpu hog? From the lack of many post concerning this problem i'd have to think no. So i'm wondering if it is something to do with my computer config.

To that extent, i've disabled my Anti-virus (Avast) and killed all non-essential processes running and tried calibre, but its the same problem. (I never have any indexing programs running without consent.)

I've noticed that the mem usage can reach ~500,000k, which is half of my total physical mem, when the app window is selected. Currently it is over 200,000k and it hasn't been the selected app for at least 10 minutes.

In general the app runs sluggish and is not very responsive, i assume it is due to the cpu issue.

As for drivers, i keep them pretty up to date. For instance, i can play Dragons Age with no video or sound issues.

Does anyone have a solution or a suggestion? It would be appreciated as this app, aside from the problem i'm having, seems to be pretty nice, and would be immensely useful if it operated correctly.

My system specs are listed below:

Summary

--------------------------------------------------------------------------------


Computer:
Operating System Microsoft Windows XP Professional
OS Service Pack Service Pack 3
DirectX 4.09.00.0904 (DirectX 9.0c)
Computer Name ******
User Name ******

Motherboard:
CPU Type AMD Athlon 64, 2000 MHz (10 x 200) 3000+
Motherboard Name ECS 755-A2 (5 PCI, 1 AGP, 1 CNR, 2 DDR DIMM, Audio, LAN)
Motherboard Chipset SiS 755, AMD Hammer
System Memory 1024 MB (PC3200 DDR SDRAM)
BIOS Type Award (09/16/05)
Communication Port Communications Port (COM1)
Communication Port ECP Printer Port (LPT1)

Display:
Video Adapter NVIDIA GeForce 7600 GS (Omega 2.169.21) (512 MB)
Monitor Plug and Play Monitor [NoDB] (L74090383840)

Multimedia:
Audio Adapter SiS 7012 Audio Device

Storage:
IDE Controller SiS PCI IDE Controller
IDE Controller Standard Dual Channel PCI IDE Controller
SCSI/RAID Controller ASU7C78W IDE Controller
Disk Drive Maxtor 6Y080L0 (80 GB, 7200 RPM, Ultra-ATA/133)
Disk Drive Maxtor 6L200M0 (200 GB, 7200 RPM, SATA)
Disk Drive ST310005 20AS USB Device (931 GB, USB)
Optical Drive DVDRW IDE1108 (DVD+RW:8x/4x, DVD-RW:8x/4x, DVD-ROM:12x, CD:40x/24x/40x DVD+RW/DVD-RW)
SMART Hard Disks Status OK

Partitions:
C: (NTFS) 130002 MB (106170 MB free)
D: (NTFS) 49999 MB (31922 MB free)
E: (NTFS) 78159 MB (36109 MB free)
G: (NTFS) 953867 MB (522956 MB free)
Total Size 1183.6 GB (680.8 GB free)

Input:
Keyboard HID Keyboard Device
Mouse HID-compliant mouse

Network:
Network Adapter SiS 900-Based PCI Fast Ethernet Adapter (192.168.0.10)

Peripherals:
Printer Lexmark 3100 Series
Printer Microsoft XPS Document Writer
USB1 Controller SiS 7001 PCI-USB Open Host Controller
USB1 Controller SiS 7001 PCI-USB Open Host Controller
USB1 Controller SiS 7001 PCI-USB Open Host Controller
USB2 Controller SiS 7002 USB 2.0 Enhanced Host Controller
USB Device Generic USB Hub
USB Device Lexmark 3100 Series
USB Device USB Composite Device
USB Device USB Human Interface Device
USB Device USB Human Interface Device
USB Device USB Human Interface Device
USB Device USB Mass Storage Device
USB Device USB Printing Support

[desertgrandma]

Welcome to MobileRead, ozzy.

Someone should post soon with suggestions.

[kovidgoyal]

That's definitely not normal behavior. As for suggestions on what to do about it, I can't really think of anything beyond killing antivirus and other processes (malware scanners, windows defender and such) that might be interfering.

One user recently reported a similar problem on OS X, but was able to resolve it by stopping some processes that were interfering with calibre.

[Manichean]

I'm guessing here, but might that have to do with the way Python is executed on windows? If I remember correctly, Python programs are executed similarly to Java programs in that they rely on an interpreter to be present on the system. And if I remember correctly, most *nix OSes have an interpreter in their standard install, which windows doesn't, so Calibre has to bring it's own, which doesn't seem to be as fast. Does that make sense to anyone?

That being said, I've never tried importing hundreds of books, but I'd kind of expect the program to use whatever CPU time it could get its hands on to do tasks like importing or converting. On the other hand, taking hours to import books does seem to be a very long time, and 60-90% utilization when just running the UI is excessive, but, I believe, not a programming fault, but caused by the choice of Python as a programming language.

And lastly, for what it's worth, my Calibre installation (with a library of about 200 books) takes anywhere from 10-30% CPU when just jumping from one book to another. My system specs are roughly equal to yours.

[sherya mathur]

by using the anti virus we find that our CPU is working fastly and the CPU utilization is done through synchronization.

[theducks]

V 6.40 Even when sending files to my device (save to disk), I found that my A/V was being hit heavily .

I had excluded the Device drives (I locked the letters, and verified they were still correct).
I excluded the library.
I (unhappily, excluded a live program) C:... /calibre2
No Joy
Calibre is using other file-space somewhere that triggers a real time scan of it'sactivity.

Disabling A/V was the only thing that dropped the usage and really improved performance.

[kovidgoyal]

You forgot to exclude the temp files directory, but of course, if you disable that, there's no point any longer in having the A/V at all.

[itimpi]

You might also need to exclude the calibre-parallel processes which do a lot of the work.

[KevinH]

Hi,

I see the same high cpu usage and similar problems using the latest versions under Mac OSX Snow Leopard.

I opened Activity Monitor and looked and it seems that the main calibre.app seems to be running under Rosseta (PowerPC) on my Intel Mac while all of the calibre-parallel threads are Intel Native.

Under Snow Leopard, it no longer gives me the option to disable or enable Rosetta for the calibre.app so I could not change this.

When a conversion is done using the gui, a new thread (running) intel is spawned (but none of the calibre-parallel threads are used) and this new calibre intel thread sucks up 100% cpu for a long time.

So somehow, I think this problem is caused by the universal binaries and the use of Rosseta (the PowerPC code conversion)

I keep Rosseta installed because I need to still use the PowerPC version of Microsoft Office 2004 to get full Macro support.

Any ideas here, how I go about forcing calibre.app to run as a native Intel program on my Mac as opposed to running in Rosseta (PowerPC).

Thanks

KevinH

[theducks]

Originally Posted by kovidgoyal

You forgot to exclude the temp files directory, but of course, if you disable that, there's no point any longer in having the A/V at all.

There are certain paths I will not exclude
If I have any doubts, I surf from my Linux box.

[KevinH]

Hi,

This is so frustrating. calibre.app (according to my Activity Monitor) is the only PowerPC program I have running under MacOSX Snow Leopard x86_64.

For some reason, no matter what you set for LSRequiresNative or LSArchitecturePriority in the Info.plist, it ALWAYS launches as PowerPC.

I disable Rosetta completely but all that did was make calibre.app hang upon startup.

Something in that binary is requiring PPC to start up.

Any ideas on how to strip the frameworks in calibre.app of all ppc code to force it to come up native is welcome.

Thanks,

KevinH

[kovidgoyal]

There's nothing in the binaries that forces PowerPC, calibre runs as Intel on my Intel Mac. But, if you want to strip powerpc code use lipo

[KevinH]

Okay,

I have no idea why the Info.plist entries are being ignored in Snow Leopard for calibre.app. Literally the only way I could get calibre.app to open up as an Intel (and not use Rosetta) on my Intel Mac was to do the following:

cd /Applications/calibre.app/Contents/MacOS
lipo -remove ppc calibre -output calibre.new
mv calibre.new calibre


Now, the program launches in half the time, the interface actually works properly, the calibre-parallel thread is used to do the work (unlike before). It still hits 100% of its cpu but on a multiprocessor system, it is just the one thread/cpu and the system stays quite responsive.

I am not sure why but from now on, I am going to strip out the ppc code from calibre so that it actually runs properly on my machine.

Just in case others run into the same thing.

Hope this helps,

Kevin

[KevinH]

Hi,
By the way in my home directory under Library/Preferences/ there is a binary plist called com.apple.LaunchServices.plist and it had a net.kovidgoyal.calibre architecture entry that was set to PPC. Perhaps some time in the past I used getinfo to set the architecture to ppc, and after upgrading to Snow Leopard, there was not way to disable that.

I used the plist editor to delete that entry.

Hopefully if anyone else runs into this, they can see if it helps them versus using lipo to strip things.

Thanks for your help.

Kevin

[jkzfixme]

I got a Kindle 3, and through various postings found Calibre to be the only real player in the ebook field. I was actually pretty OK with that, as I have heard many favorable things about Calibre. Setup was painless and it immediately recognized my kindle 3. The problem started when I tried to import my relatively large Ebook collection. I am trying to import 5,000 + or - at a time.

While it is importing it pegs the processor .... I mean everything it can get, as well as maxing out the memory. I have a rather extensive collection of books and that was only one folder. The problem being that it took 3 days to import those, and I have 6-7 other folders I would like in the database with similar size and structure.

My system is relatively old but still pretty snappy

AMD Phenom(tm) 9600 Quad-Core Processor
4 Gb DDR2
2.6.35-24-generic #42-Ubuntu SMP x86_64 GNU/Linux
Python 2.6.6
Calibre 0.7.18

Any words from the wise would be greatly appreciated !