Calibre, OS X, and endless disk grinding

thibaulthalpern · 04-25-2009, 02:36 PM

Is anyone using the latest versions of Calibre on OS X and experiencing disk grinding (a lot!) after calibre has been open for a short while?

I've been keeping calibre open for say about 5 to 10 minutes while I add my PDFs to calibre. Sometimes, calibre is sitting in the background while I'm in Acrobat Professional fiddling with the PDF before it is added (or after it has been added). I notice that after a while there is a lot of disk grinding going on. I couldn't figure out what program was doing that and then I finally just quit Calibre and the grinding stopped. Is Calibre the culprit?

One thing of note: 99.99% of my digital books are PDFs. I wonder if there is still an ongoing issue between Calibre and PDFs.

Evi1d33d · 04-25-2009, 03:12 PM

Grinding is a broad term, it could be simple disk activity or disk failure.

thibaulthalpern · 04-25-2009, 03:32 PM

Of course grinding is a broad term and "precisely" because all I know is that there is a lot of disk activity. I don't think my disk is failing because it has stopped and also the S.M.A.R.T status is normal. Neither of these are foolproof but chances are it isn't failing.

sirbruce · 04-25-2009, 04:02 PM

How much free memory do you have? Calibre can consume 125MB+ on my Windows machine; if I've got two conversion processes going at the same time that's 3x that much. Although under OS X those should be copy-on-write forks, so duplicate pages shouldn't be consuming extra memory. But maybe that's not how they work these days. Anyway, point is, if you're low on free memory then using Calibre can cause a lot of disk paging.

thibaulthalpern · 04-25-2009, 04:52 PM

I have 2GB of RAM. Usually three programmes are opened--Safari (web browser), Calibre, and Acrobat Professional. I've had the disk thrashing happen on and off since maybe 0.5.8. It doesn't come all the time but it does seem to come every now and then. It's gone for now.

kovidgoyal · 04-25-2009, 05:19 PM

when it starts showing high disk activity does it taper off after a while or does it stay high? And do you have the content server running? Or scheduled news downloads?

thibaulthalpern · 04-25-2009, 05:32 PM

Quote:

Originally Posted by kovidgoyal

when it starts showing high disk activity does it taper off after a while or does it stay high? And do you have the content server running? Or scheduled news downloads?

I don't know if the disk activity tapers off because what happens is that my computer gets increasingly slow (to a point) and then not as responsive. So I end up quitting Calibre.

I don't have content server running and I don't have scheduled news downloads.

I think this has happened maybe three times or so in the recent past. I will pay attention to it closely if it happens again. I don't know what triggers it, and the it may not even be Calibre but it could be.

Typically I'm adding books to Calire (I have a ton of books I need to add and update the metadata). Sometimes when editing the metadata, I open up the PDF in Acrobat Professional for some info (read bits of it to figure out how to categorise it etc.). Sometimes I may "optimise" the PDF after I've added the PDF to Calibre. Optimise basically means reducing the file size and removing unnecessary content. It seems that somehow some combination of these activities may start disk thrashing (done by Calibre?).

I have 2GB of RAM, and 30.24GB of free space on my internal drive and 34.8GB of free space on my external drive where the data folder for Calibre resides.

kovidgoyal · 04-25-2009, 05:39 PM

Basically if you dont have the content server running or scheduled downloads, the only thing calibre should be doing is running a few timers to check various conditions, this should not cause any disk activity at all.

sirbruce · 04-25-2009, 05:47 PM

Calibre has had some PDF issues but I assume all of those are addressed in the latest release?

What you've described sounds very much like the computer running out of RAM and having to rely on disk paging. I imagine Adobe Professional has a pretty large footprint, and so do web browsers if you have open a lot of tabs. Combine that with a few Calibre processes and the OS and I wouldn't be surprised OS X starts hammering the disk.

It may be that Calibre has a memory leak where memory allocated when you import a new book into the database is not immediately released. So over time, as you add more and more books, the Calibre process consumes more and more memory until thinks reach a tipping point. Shutting down Calibre completely and restarting it would clear up that issue.

thibaulthalpern · 04-25-2009, 07:50 PM

Quote:

Originally Posted by sirbruce

Calibre has had some PDF issues but I assume all of those are addressed in the latest release?

What you've described sounds very much like the computer running out of RAM and having to rely on disk paging. I imagine Adobe Professional has a pretty large footprint, and so do web browsers if you have open a lot of tabs. Combine that with a few Calibre processes and the OS and I wouldn't be surprised OS X starts hammering the disk.

It may be that Calibre has a memory leak where memory allocated when you import a new book into the database is not immediately released. So over time, as you add more and more books, the Calibre process consumes more and more memory until thinks reach a tipping point. Shutting down Calibre completely and restarting it would clear up that issue.

Of these, the last scenario is more likely than the other ones you've listed. I've used Acrobat Professional and Safari extensively and no problems.

I'll report back if there are additional developments.

sirbruce · 04-25-2009, 07:52 PM

The last thing I listed was not a "scenario"; it was an explanation of how Calibre could get big enough to cause disk activity, but which would strike intermittently based on intensity of usage. This is not a matter of "no problems"; this is perfectly normal computer behavior when you have a lot of things running at once taking up a lot of memory.

zartemis · 05-23-2010, 02:14 AM

I can report a similar problem, but with Windows XP. I installed Calibre 0.6.54 and prior to importing any books at all and changing no default settings (nor starting the webserver), I get regular disk activity every second. In my case it starts immediately and at least goes for several minutes (annoying enough that I haven't let it run longer than that to see if it will stop). Exiting Calibre stops the activity immediately.

I tried rebooting and running only Calibre, same regular disk noise.

I used to run an old version of Calibre on this machine, but had reinstalled windows since then (version 0.4.x or so). Just to make sure no monkey business with old install files, I uninstalled Calibre, searched for calibre folders and removed an Appdata calibre folder and reinstalled Calibre. Same disk noise whenever it runs.

Willing to provide more info/debug help if given directions on what to check for.

04-25-2009, 02:36 PM	#1
thibaulthalpern Evangelist Posts: 478 Karma: 451808 Join Date: Feb 2009 Location: California, USA Device: my two eyes, KLiiK, Sony PRS-700	Calibre, OS X, and endless disk grinding Is anyone using the latest versions of Calibre on OS X and experiencing disk grinding (a lot!) after calibre has been open for a short while? I've been keeping calibre open for say about 5 to 10 minutes while I add my PDFs to calibre. Sometimes, calibre is sitting in the background while I'm in Acrobat Professional fiddling with the PDF before it is added (or after it has been added). I notice that after a while there is a lot of disk grinding going on. I couldn't figure out what program was doing that and then I finally just quit Calibre and the grinding stopped. Is Calibre the culprit? One thing of note: 99.99% of my digital books are PDFs. I wonder if there is still an ongoing issue between Calibre and PDFs.

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Calibre 0.7.22, Empty eBooks not saved on Save to disk?	Nadrek	Calibre	3	10-04-2010 12:24 PM
Calibre will not save to disk	mrbillb	Calibre	0	09-10-2010 06:56 PM
Saving to disk, Calibre, epub & covers	varaonaid	Calibre	5	05-03-2010 01:27 PM
Save to Disk: How to use Calibre with Windows Mobile devices?	Starson17	Calibre	0	12-27-2009 09:56 PM
Can't SAVE TO DISK Calibre 0.6.24	Rootman	Calibre	5	12-06-2009 05:01 AM

04-25-2009, 03:12 PM	#2
Evi1d33d Enthusiast Posts: 47 Karma: 14 Join Date: Sep 2007 Device: Red PRS-505	Grinding is a broad term, it could be simple disk activity or disk failure.

04-25-2009, 03:32 PM	#3
thibaulthalpern Evangelist Posts: 478 Karma: 451808 Join Date: Feb 2009 Location: California, USA Device: my two eyes, KLiiK, Sony PRS-700	Of course grinding is a broad term and "precisely" because all I know is that there is a lot of disk activity. I don't think my disk is failing because it has stopped and also the S.M.A.R.T status is normal. Neither of these are foolproof but chances are it isn't failing.

04-25-2009, 04:02 PM	#4
sirbruce Provocateur Posts: 1,859 Karma: 505847 Join Date: Feb 2009 Location: Columbus, OH Device: Kindle Touch, Kindle 2, Kindle DX, iPhone 3GS	How much free memory do you have? Calibre can consume 125MB+ on my Windows machine; if I've got two conversion processes going at the same time that's 3x that much. Although under OS X those should be copy-on-write forks, so duplicate pages shouldn't be consuming extra memory. But maybe that's not how they work these days. Anyway, point is, if you're low on free memory then using Calibre can cause a lot of disk paging.

04-25-2009, 04:52 PM	#5
thibaulthalpern Evangelist Posts: 478 Karma: 451808 Join Date: Feb 2009 Location: California, USA Device: my two eyes, KLiiK, Sony PRS-700	I have 2GB of RAM. Usually three programmes are opened--Safari (web browser), Calibre, and Acrobat Professional. I've had the disk thrashing happen on and off since maybe 0.5.8. It doesn't come all the time but it does seem to come every now and then. It's gone for now.

04-25-2009, 05:19 PM	#6
kovidgoyal creator of calibre Posts: 43,860 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	when it starts showing high disk activity does it taper off after a while or does it stay high? And do you have the content server running? Or scheduled news downloads?

04-25-2009, 05:39 PM	#8
kovidgoyal creator of calibre Posts: 43,860 Karma: 22666666 Join Date: Oct 2006 Location: Mumbai, India Device: Various	Basically if you dont have the content server running or scheduled downloads, the only thing calibre should be doing is running a few timers to check various conditions, this should not cause any disk activity at all.

04-25-2009, 05:47 PM	#9
sirbruce Provocateur Posts: 1,859 Karma: 505847 Join Date: Feb 2009 Location: Columbus, OH Device: Kindle Touch, Kindle 2, Kindle DX, iPhone 3GS	Calibre has had some PDF issues but I assume all of those are addressed in the latest release? What you've described sounds very much like the computer running out of RAM and having to rely on disk paging. I imagine Adobe Professional has a pretty large footprint, and so do web browsers if you have open a lot of tabs. Combine that with a few Calibre processes and the OS and I wouldn't be surprised OS X starts hammering the disk. It may be that Calibre has a memory leak where memory allocated when you import a new book into the database is not immediately released. So over time, as you add more and more books, the Calibre process consumes more and more memory until thinks reach a tipping point. Shutting down Calibre completely and restarting it would clear up that issue.

04-25-2009, 07:52 PM	#11
sirbruce Provocateur Posts: 1,859 Karma: 505847 Join Date: Feb 2009 Location: Columbus, OH Device: Kindle Touch, Kindle 2, Kindle DX, iPhone 3GS	The last thing I listed was not a "scenario"; it was an explanation of how Calibre could get big enough to cause disk activity, but which would strike intermittently based on intensity of usage. This is not a matter of "no problems"; this is perfectly normal computer behavior when you have a lot of things running at once taking up a lot of memory.

05-23-2010, 02:14 AM	#12
zartemis Addict Posts: 283 Karma: 2002 Join Date: Nov 2007 Device: Kindle3, Kindle2, Nook, NookColor	I can report a similar problem, but with Windows XP. I installed Calibre 0.6.54 and prior to importing any books at all and changing no default settings (nor starting the webserver), I get regular disk activity every second. In my case it starts immediately and at least goes for several minutes (annoying enough that I haven't let it run longer than that to see if it will stop). Exiting Calibre stops the activity immediately. I tried rebooting and running only Calibre, same regular disk noise. I used to run an old version of Calibre on this machine, but had reinstalled windows since then (version 0.4.x or so). Just to make sure no monkey business with old install files, I uninstalled Calibre, searched for calibre folders and removed an Appdata calibre folder and reinstalled Calibre. Same disk noise whenever it runs. Willing to provide more info/debug help if given directions on what to check for.

Advert

Advert