Calibre slow startup

[ericmjones]

Running Calibre 2.20 on windows 7 64 bit. dual core processor, 8gb memory and Library ~9000 books. Suddenly Calibre is taking a long time to load. The splash screen comes up in a few second - but the library takes a further 2 minutes to load.
During this time the task manager shows calibre using ~2% of the cpu (intermittently) with memory increasing in steps to 207,000K.
Lots of disk activity during this period.

I tried switching off the anti-virus but it makes no difference. This only happens on the first activation of calibre after starting the computer. all subsequent starts of calibre are within a few seconds.

Any thoughts??

[theducks]

You tried the usual suspect

When was the HD last defragged?

[BetterRed]

Originally Posted by ericmjones

Running Calibre 2.20 on windows 7 64 bit. dual core processor, 8gb memory and Library ~9000 books. Suddenly Calibre is taking a long time to load. The splash screen comes up in a few second - but the library takes a further 2 minutes to load.
During this time the task manager shows calibre using ~2% of the cpu (intermittently) with memory increasing in steps to 207,000K.
Lots of disk activity during this period.

I tried switching off the anti-virus but it makes no difference. This only happens on the first activation of calibre after starting the computer. all subsequent starts of calibre are within a few seconds.

Any thoughts??

@ericmjones - if you start calibre very soon after you start Windows, then the delay may be due to Windows (or some other 3rd party program) still doing its start up tasks - eg creating a restore point. I've also noticed that the indexer sometimes finds things to do at start up, but if its set to run at low priority (which is the default) it should yield to higher priority processes.

If above applies try waiting 10 minutes before starting calibre -- just to see what happens -- not suggesting that as a 'real' solution.

IMO running defrag on Win 7/NTFS is a waste of time, especially with the standard MS defragger. Defragging is done dynamically by Windows 7 when there's nothing else happening, unless its been turned off via the Policy Editor or a Registry hack. I believe some of the pay4me defraggers can make a difference on busy servers.

Are you sure that its calibre that's generating the disk i/o. Task Manager and tools like Process Explorer can help determine that.

Also calibre 2.20 had a fixup release, so you might want to try a fresh download and install from here ==>> Download calibre

One thing to try is to use Library->Remove library to 'detach' your library from calibre (don't worry it won't delete it as such) reboot and start calibre - if its still slow then that would 'indicate' that the disk i/o it not calibre rattling around in your library. You can 're-attach' the library via the Library->Create/Switch->Use existing library option

And you could also try a downgrade to a previous version. I'd suggest you backup first even though its not theoretically necessary, see ==>> How do I backup calibre?

What happens if you boot into Safe mode?

Also try starting calibre with the --ignore-plugins switch, see ==>> calibre command

Columns made from other columns have been known to generate slow calibre starts, but I am pretty certain they affect all calibre starts - not just the first time.

BR

[chaley]

Originally Posted by BetterRed

IMO running defrag on Win 7/NTFS is a waste of time, especially with the standard MS defragger. Defragging is done dynamically by Windows 7 when there's nothing else happening, unless its been turned off via the Policy Editor or a Registry hack.

Can you provide sources for this? As far as I can tell, by default W7 defrags on Wednesday at 01:00, if the machine is running or can be awakened. On my machines, defrag appears to almost never run automatically.

[BetterRed]

Originally Posted by chaley

Can you provide sources for this? As far as I can tell, by default W7 defrags on Wednesday at 01:00, if the machine is running or can be awakened. On my machines, defrag appears to almost never run automatically.

Whoops confusion - it's my old NT4 server that defrags when its idle, for some reason my typing brain mistakenly thought the workstations were the same.

My workstations defrag dally at 11:00. Yes, I just read the default for Win 7 is Wed 01:00, so I must have changed it back in 2009.

From what I can see the schedule works providing the workstation is on. The 2 drives on this w/s report they were defragged today, one at 11:03 the other at 11:11, it's currently 14:00ish. When the defrag starts the only way I can discern it's running is by the disk lights, not by responsiveness. It takes a few minutes to do 2x2TB WD Caviar blacks. The workstation also has a 256GB SSD system drive. The WD drives have hefty caches and they're no where near full.

I don't defrag external drives, they're also 2TB WD Caviar Blacks. When they're in use I mount them in a disk dock that connects to a PCIe USB 3.0 hub. My observation is that they're faster than the same drives on SATA 2. Again they have hefty caches and they're not too full - and they are compressed.

I'm just not convinced defragging makes a great deal of difference on modern drives with hefty caches, providing the drives are not close to full.

BR

PS - I just recalled, a friend recently bought her Vista laptop for me to have a look at. It started going very slowly after she ran Vista defrag (first time in 7+ years). I managed to speed it up a bit, but it really needs a fresh Windows install - I'm awaiting arrival of Vista Business product. Apparently MS tore their VISTA iso's down last year

Why did she do a defrag - because someone screwed around with her email accounts and got her confused, so she 'thought a defrag might help'

[theducks]

Originally Posted by BetterRed

Whoops confusion - it's my old NT4 server that defrags when its idle, for some reason my typing brain mistakenly thought the workstations were the same.

My workstations defrag dally at 11:00. Yes, I just read the default for Win 7 is Wed 01:00, so I must have changed it back in 2009.

From what I can see the schedule works providing the workstation is on. The 2 drives on this w/s report they were defragged today, one at 11:03 the other at 11:11, it's currently 14:00ish. When the defrag starts the only way I can discern it's running is by the disk lights, not by responsiveness. It takes a few minutes to do 2x2TB WD Caviar blacks. The workstation also has a 256GB SSD system drive. The WD drives have hefty caches and they're no where near full.

I don't defrag external drives, they're also 2TB WD Caviar Blacks. When they're in use I mount them in a disk dock that connects to a PCIe USB 3.0 hub. My observation is that they're faster than the same drives on SATA 2. Again they have hefty caches and they're not too full - and they are compressed.

I'm just not convinced defragging makes a great deal of difference on modern drives with hefty caches, providing the drives are not close to full.

BR

PS - I just recalled, a friend recently bought her Vista laptop for me to have a look at. It started going very slowly after she ran Vista defrag (first time in 7+ years). I managed to speed it up a bit, but it really needs a fresh Windows install - I'm awaiting arrival of Vista Business product. Apparently MS tore their VISTA iso's down last year

Why did she do a defrag - because someone screwed around with her email accounts and got her confused, so she 'thought a defrag might help'

Defrag the Hive and cache using the tool from sysinternals

[BetterRed]

@chaley - this afternoon I checked 3 Win 7 systems at my local community centre. Each has the standard 01:00 hrs on Wednesday setting, they report the most recent defrag occurred last Wednesday (Mar 4th) at 08:20, 09:05 and 09:41 - which is a few minutes after they were booted.

So 5 of 5 systems I've checked down here do the defrag as per schedule, on my two boxes daily at 11:00am, and on the other three on Wednesdays soon after boot up.

Another way to disable auto defrag -- other than unchecking the Run on a schedule in the Defragger -- is to disable the service. I've seen that on a few systems, my guess is that Windows was installed like that at the factory. Or one could add a policy into the GP settings - that's a bit trickier, there were some videos at technet that showed how.

I've yet to see a brand name OEM Windows 7 install that IMO ought not be redone - a) to get rid of vendor bloat and free trials, b) to enable services that IMO ought not be disabled - indexing, link tracking, auto-defrag... etc.

I always encourage friends to buy a full Windows license and reinstall themselves or ask me to do it for them - under Australian law that won't invalidate warranty. Often as not the retailer will throw in the full license for not very much - eg Win7 Pro for $50.

But I suspect you know all that

BR

[chaley]

@BR: thanks. I looked at my machines and indeed they all defrag on Wed morning when they are turned on.

Why do you need to buy a full license? Doesn't the machine come with an OEM license valid for it? Don't you just need standard installation media (easy to get) and the installation's product key?

[BetterRed]

I'm glad you sorted that out - its one of those things I just set & forgot. I probably changed to 11:00 because a) I don't like to 'overload' the system at start up, and b) Wednesday down here is Tuesday over there, so it would clash with Patch Tuesday updates.

Why buy a full license ? Future proofing.

You can't do an inline re-install with many OEM vendor patched (lashed up) media, and I don't trust 3rd party streams. Also if you're a few hundred km from a shop and at the end of a flaky 256kbit sat link (try downloading a 3.5GB iso at that speed) you need all the help you can get - like a shiny gold DVD with a Microsoft brand in the bottom drawer. And most of my friends aren't exactly paupers.

BR

[eschwartz]

While some people might find it easier to deal with a shiny Microsoft-delivered install media, the ISOs from Digital River should be fine for the slightly more adventurous type. Or indeed the type that has better internet.

Oh, wait -- MS pulled those down recently.

Still, if they can find a reliable mirror -- or do what I did and download random suggested sources then compare sha1sums posted on TechNet -- then you should get valid official Win7 install media. Gone are the WinXP days, supposedly, when retail media wouldn't use OEM keys.

I certainly don't have the money to waste -- especially when I am doing this as a favor to a friend (has no money either ) who just nuked his laptop trying a Reset To Factory Defaults. After I installed a dual-boot Linux Mint (his request). Somehow it trampled all over the HDD.

[SquiddOhio]

Even though my machine is pretty fast, some large programs (Photoshop) and some programs like Calibre that load a large database can take some time to get up and running. I solved that problem by replacing my C: drive with a Solid State Drive. I put the Calibre library on there also. It's a drastic solution, but the computer boots up in 20 seconds or so, and the price of SSDs is now reasonable. Highly recommended.

[ericmjones]

Originally Posted by BetterRed

@ericmjones - if you start calibre very soon after you start Windows, then the delay may be due to Windows (or some other 3rd party program) still doing its start up tasks - eg creating a restore point. I've also noticed that the indexer sometimes finds things to do at start up, but if its set to run at low priority (which is the default) it should yield to higher priority processes.

If above applies try waiting 10 minutes before starting calibre -- just to see what happens -- not suggesting that as a 'real' solution.

IMO running defrag on Win 7/NTFS is a waste of time, especially with the standard MS defragger. Defragging is done dynamically by Windows 7 when there's nothing else happening, unless its been turned off via the Policy Editor or a Registry hack. I believe some of the pay4me defraggers can make a difference on busy servers.

Are you sure that its calibre that's generating the disk i/o. Task Manager and tools like Process Explorer can help determine that.

Also calibre 2.20 had a fixup release, so you might want to try a fresh download and install from here ==>> Download calibre

One thing to try is to use Library->Remove library to 'detach' your library from calibre (don't worry it won't delete it as such) reboot and start calibre - if its still slow then that would 'indicate' that the disk i/o it not calibre rattling around in your library. You can 're-attach' the library via the Library->Create/Switch->Use existing library option

And you could also try a downgrade to a previous version. I'd suggest you backup first even though its not theoretically necessary, see ==>> How do I backup calibre?

What happens if you boot into Safe mode?

Also try starting calibre with the --ignore-plugins switch, see ==>> calibre command

Columns made from other columns have been known to generate slow calibre starts, but I am pretty certain they affect all calibre starts - not just the first time.

BR

Disk is defragged regularly (done in the last few days) Running 2.20. Removed library as you suggested and then restarted calibre and opened library - back to normal opening in a few seconds. Looks like a "funny" that has been cleared. Restarted machine and it still opens quickly. Thanks for the ideas.

[BetterRed]

Originally Posted by ericmjones

Disk is defragged regularly (done in the last few days) Running 2.20. Removed library as you suggested and then restarted calibre and opened library - back to normal opening in a few seconds. Looks like a "funny" that has been cleared. Restarted machine and it still opens quickly. Thanks for the ideas.

- that wasn't supposed to fix the problem

Curious - was the library folder ever 'attached' via a symbolic link or a junction ?

BR

[ericmjones]

Originally Posted by BetterRed

- that wasn't supposed to fix the problem

Curious - was the library folder ever 'attached' via a symbolic link or a junction ?

BR

No - directly open when I first used Calibre. survived several calibre upgrades with no problems. I cannot understand it either

[BetterRed]

@ericmjones - I'll add 'detach/re-attach library' to my list of "try this" solutions. I normally have an inkling of why "this" is worth "trying" - but on this occasion I haven't a clue.

Some sort of ephemeral 'hook up' between calibre, your library, and windows or something; that was 'torn down' by detaching the library from calibre - weird

I wonder if something to do with the 'lazy writing' of book folder .opf files, or Find Duplicates Marked Exemptions went haywire - I can 'vaguely imagine' that a detach/attach library sequence might overcome such a glitch.

BR