Very slow, lots of resources used when starting program

[MidwestJen]

The short and dirty version:
My computer is now taking approximately 7 minutes to open a database of ~20k books with 3 custom columns (one concatenates other columns, one pulls a series index, and one assigns tag like information on what virtual library the item is in). I have made no changes to those columns and how they calculate for months. When opening calibre, my memory and disk resources are almost maxed out until it finally opens fully. Is there a way to find out what is going on in the background?

The long and detailed version:
My Windows 10 computer recently updated. After the update I noticed that when opening calibre, it was taking a long time (timed later at 7 min). I will admit that I don't remember how long it took before this, but it wasn't long enough to cause real concern and honestly I don't open and close it very often. But I've had this problem before, and it usually has to do with my custom columns. I've got 3, but two of them are fairly simple (concatenating other columns and pulling out a series index number). The third assigns tag like information on what virtual libraries the item is in. That one tends to be my problem child since my library has around 20k items and I've currently got 11 virtual libraries. It seems to exaggerate any other issues imo.

Anyway - after reviewing what I had going on, I first decided to update calibre (I hadn't updated in a few versions). That didn't work, it still took a long time to open. Then I decided to remove the mark duplicates plugin. It hasn't been updated in forever and every time it opens it marks my old duplicates - which is a problem because one of my virtual libraries is assigned using (marked:true). I removed the plugin and restarted calibre - still a long time to open. I rebooted, started task manager, and then started calibre while hand logging what was going on. It took 7 minutes, maxed out my memory (12gb) and disk at various times and only released the resources when it fully opened. Another iteration of rebooting/restarting confirmed those results, only it took a bit over 8 minutes that time.

I don't have any other issues really. The program itself is working well, it just takes an age to open (and I can't really do much else while it's working on that). Is there a way to find out in the background what's actually causing this? It's not an emergency, I don't open and close this often, but I would like to have it open a bit more smoothly if possible.

Calibre Version: 2.47, 64bit
Plugins: Count Pages, EpubMerge, FanFicFare, Generate Cover; metadata source plugins for FictionDB, B&N; and a plug in for DRM books - all others are standard installation

[ShellShock]

Maybe disable all the plugins, see if that fixes it, and then reenable them one by one?

[PeterT]

Antivirus scanning files?

[BetterRed]

Originally Posted by ShellShock

Maybe disable all the plugins, see if that fixes it, and then reenable them one by one?

No need to initially 'disable' all the PIs, just run calibre using "calibre --ignore-plugins" command, in Windows I suggest from the Run program popup (Winkey+R). Also, some optional PIs cannot be Disabled via the Preferences->Plugins->Disable/Enable plugin button; to achieve the same result they must be Removed.

@MidwestJen - whilst waiting for calibre to start can you do non-calibre tasks - e.g. use a browser, word processor, play/edit media files etc.

for developers - would an '--ignore-custom-columns' option be feasible?

BR

[MidwestJen]

I'm not at my computer atm, so can't try the disabling of plugins yet, but thought I'd get an answer out there for this stuff.

I haven't updated any plugins today (or within the last week or two), but I'll certainly try disabling and see if the problem is there. It shouldn't be antivirus as that hasn't changed either, but I can disable Norton for a brief time and verify.

@BetterRed - Yes, I can do other things while calibre is loading, although usually that consists of browsing via Firefox. It's sluggish and at times needs more than a few seconds to process what is usually instantaneous, but my system isn't locking completely up.

I was hoping for a debug command that logged what is actually happening in the background - so it would show that x time was used for x part of loading the program. Would be easier to troubleshoot if there was one, even if I only understood a fraction of what it said.

[theducks]

Originally Posted by MidwestJen

I'm not at my computer atm, so can't try the disabling of plugins yet, but thought I'd get an answer out there for this stuff.

I haven't updated any plugins today (or within the last week or two), but I'll certainly try disabling and see if the problem is there. It shouldn't be antivirus as that hasn't changed either, but I can disable Norton for a brief time and verify.

@BetterRed - Yes, I can do other things while calibre is loading, although usually that consists of browsing via Firefox. It's sluggish and at times needs more than a few seconds to process what is usually instantaneous, but my system isn't locking completely up.

I was hoping for a debug command that logged what is actually happening in the background - so it would show that x time was used for x part of loading the program. Would be easier to troubleshoot if there was one, even if I only understood a fraction of what it said.

W10 x64 10K+ books about 15sec for the GUI with 4 coloring rules, NO columns built from other columns.

Exclude your Library FOLDER from your A/V, the books have been scanned already

Disable Cloud sync on your Library folder.. And never run from a Network share

[FizzyWater]

I seem to remember this came up in another thread awhile back, and it ended up having to do with there being a large number of downloaded, but not yet installed, Windows Update files (for those with settings "download, but let me decide", if I remember correctly).

The thread was started by Harry....if I can find it, I'll edit this reply with a link.

EDIT: Okay, the thread I'm thinking of was about the computer completely freezing up when Calibre started. You might want to look this thread over, there were lots of suggestions, including the one to clean out the Windows Update files (which is what I had to do to get my problem to clear out).

[theducks]

Fizzy gave me a related idea.
Some recent W10 updates needed 2 reboots to complete the load

[BetterRed]

Originally Posted by MidwestJen

The third assigns tag like information on what virtual libraries the item is in. That one tends to be my problem child since my library has around 20k items and I've currently got 11 virtual libraries. It seems to exaggerate any other issues imo.

@MidwestJen; I would try removing that column - After saving the definition somewhere.

BR