Is Calibre slowing down?

[Hoods7070]

At the moment I am trying to sort out my library, which is about 4500 books, and Calibre literally GROANS on my laptop (i7, 1.8GHz) and seems to have slowed down on my PC too (i7, 3.x GHz) though its speed is acceptable: simple operations that used to take microseconds now take "quite a while" - eg changing tags, copy/delete to another library etc. Other things like browsing etc. are fine.

Is this perhaps because of the addition of the Editor? Just wondering.

What would be a suggested manageable (ie in terms of speed) number of books?

[BetterRed]

@Hoods7070 - my system - i5 2.3GHz, 6Gb RAM, SATA-2 Disk, Windows 7 64-bit, Calibre 2.1 - Data and System are on separate disks - both WD Caviar Blacks.

I just copied 280 books (305 MB) from library A (~25,000 books) to library B (~5,000 books) in 2m 51sec - with no duplicate checking, that can slow things down. That's more than I would normally copy. I routinely do a library A to library B copy of 10-20 books each day in a few seconds.

How do you tag, I tag in the Book List

I would guess its something that's monitoring writes. AV and Indexers spring to mind.

FWIW - I use the MSE AV, my libraries are Windows Search indexed and the drive is compressed.

About a year ago, to 'prove' that I had my AV and Indexing optimally configured, I excluded my calibre libraries from MSE and Indexing for a few days. I could not discern any difference in performance in calibre, or anything else for that matter, so I un-excluded everything again and left the settings as they were/are.

BR

[Hoods7070]

Hmm. Thanks for the suggestions. I wonder what's doing this?? I will uninstall some stuff I put on yesterday that might be the culprit/s. On my laptop I copy/deleted 399 books of about 4500 and it took (I kid you not) ONE HOUR.

Can you tell me how to optimize indexing? I really have no idea. Not something I have bothered with much. (I use Directory Opus as my file manager and it's super fast on searches.)

Running commentary ...

I don't have check for duplicates checked. (Assuming that's set with the option in Add Files?)

I also enter tags in the book list - the only tag I have is the author's country. When I have the info I want, inc a comment "jacket", I run the book through the Modify ePub plugin. As I posted in a separate thread, unfortunately somewhere along the way about 500 tags have been lost, and comments removed. These remain the in the ebook itself, but as gone from the booklist. !@#!@$% Very, very tedious repairing all this. On the laptop just the simple process of swapping author order is taking seconds instead of being instant instead of being instant as it used to be.

For AV I use eset32 Smart Security, and have done forever, which has never presented a problem before with Calibre or anything else. The only "new" things I've installed lately are an online adblocker (AdGuard) and the latest version of Auslogics BoostSpeed - I don't like either so they were going anyway.

I am moving 600 files of 5000 books in round figures on my PC as we speak and it's about 50% through after nearly 10 minutes. I uninstalled the suspect software and added everything Calibre as exclusions in my AV (which I believe DID make a small difference) but this is still pretty slow.

I have to say there is a BIG difference between i7 1.8GHz and i7 3.whatever GHz this PC is. I've had the PC since the release of the first i7 chipsets and it has never let me down! Not impressed much with the laptop. Should have been a little less mean and bought a faster model. I hate to wait!!

I will now go and do some experimenting on the laptop. I will report back!

[BetterRed]

For Windows Search (that's content searching - its hopeless for file name searching) the main thing is to only index your data, have a look at the iFilters from Citeknet and the iFiltershop, and a get a decent iFilter for PDF - Adobe, Foxit or xChange all have them - I use the xChnage one because I use that viewer. The other thing is to run the indexing service at lowest priority you do that in the Services console.

Moving books is I/O bound. When you were moving those books your CPU was probably 90%+ idle. Do a ctrl/alt/esc, go to Performance Tab, click Resource Monitor, it gives a good overview of what's doing what.

It does seem a bit slow, are your libraries on the same drive as the system (I guess they would be on your laptop) - I keep my libraries on a second drive. Is the desktop MOBO SATA2 or SATA3, mines SATA 2 so its a bit slow.

It could be that your drive needs defragging - AFAIK Win7 defrags when its got nothing to do but I don't think its very aggressive. There are other defraggers that claim to get more oomph out of a drive, I think Pirifiorm (the cCleaner folks) might have one.

When you say the lost comments & tags are in the ebooks, do you mean they're embedded as metadata or they're in a book jacket - if they're embedded you might be able to get them back.

BR

[Agama]

I never use Windows Search to find files based on content and have therefore switched off content indexing at the drive level and applied it to all sub-folders/files, (which can take a while but it's a once only exercise.)

You can still find files by filename, this only stops content indexing.

[Adoby]

Make a backup of your calibre library and do some experimenting, and see if you can improve performance. Things to try:

Remove the tag browser.
Delete (not just hide) the formats column.
Delete (not just hide) all custom columns that depend on other columns.
Delete (not just hide) all custom columns.
Delete any rules for coloring or other dynamic behaviour.
Don't display columns you don't use.
Disable duplicate checks.

[chaley]

@Hoods7070: how much RAM does your laptop have? If not "enough", you might have started using virtual memory, making calibre use much more disk resource. How much is "enough" depends on many factors, but I would be surprised if 2GB is enough for a library that size. On a similar vein, how fast is your laptop's disk? Many laptops ship with slow-spinning drives to save power.

[hidden.platypus]

Whoa!

4500 book libraries?

I didn't realize I was that much of a neophyte.

And I was proud of of my 300 book library too.

[BetterRed]

Originally Posted by Agama

I never use Windows Search to find files based on content and have therefore switched off content indexing at the drive level and applied it to all sub-folders/files, (which can take a while but it's a once only exercise.)

You can still find files by filename, this only stops content indexing.

I define what's to be indexed via cp->indexing options (same destination different route), in there I also exclude file types - eg images, video, objects, executables etc. Basically I only index stuff I can read with my eyeballs. IIRC there are ~80,000 files indexed.

D-Opus should be good for finding by file name, I've not used it for a while but I know it was pretty slick when I did.

BR

[Hoods7070]

Originally Posted by hidden.platypus

Whoa!

4500 book libraries?

I didn't realize I was that much of a neophyte.

And I was proud of of my 300 book library too.

I'm a hoarder, what can I say? If I have one book in a series I have to have the whole series, that sort of thing. I recently did some maths and concluded that Amazon and other book sellers would collapse the stock market if I cease to buy books. I am trying to stop .... really, I am!

[Hoods7070]

Originally Posted by BetterRed

I define what's to be indexed via cp->indexing options (same destination different route), in there I also exclude file types - eg images, video, objects, executables etc. Basically I only index stuff I can read with my eyeballs. IIRC there are ~80,000 files indexed.

D-Opus should be good for finding by file name, I've not used it for a while but I know it was pretty slick when I did.

BR

]
OK - I went to Indexing Options and what do you know, MSoft doesn't even give you the option of checking/unchecking the file type list. I started doing it manually then thought !@#$ it!!

[Hoods7070]

Originally Posted by chaley

@Hoods7070: how much RAM does your laptop have? If not "enough", you might have started using virtual memory, making calibre use much more disk resource. How much is "enough" depends on many factors, but I would be surprised if 2GB is enough for a library that size. On a similar vein, how fast is your laptop's disk? Many laptops ship with slow-spinning drives to save power.

It's got 4GB, as does my PC. No doubt the laptop's disk IS slow, but the PC disks are all optimized and so on and "the best". As both systems have slowed right down with Calibre it's got to be some other issue.

[Hoods7070]

Originally Posted by Adoby

Make a backup of your calibre library and do some experimenting, and see if you can improve performance. Things to try:

Remove the tag browser.
Delete (not just hide) the formats column.
Delete (not just hide) all custom columns that depend on other columns.
Delete (not just hide) all custom columns.
Delete any rules for coloring or other dynamic behaviour.
Don't display columns you don't use.
Disable duplicate checks.

I have 7 columns in my booklist inc format (the only custom column). I deleted that and it hasn't made any significant difference - but thanks!

[FencepostErr1r]

Perhaps a bit off-topic, but Calibre feels snappier on OSX since version 2.
My library only contains 900 and some epubs, mostly fan-fiction with about three tags and some shelves.

-FencepostErr1r

[hidden.platypus]

Originally Posted by chaley
@Hoods7070: how much RAM does your laptop have? If not "enough", you might have started using virtual memory, making calibre use much more disk resource.

Originally Posted by Hoods7070
It's got 4GB, as does my PC.

I run Win 7 Home Premium 64 bit and at idle running my AV, Firewall, and, HIPS Windows reports I'm using 3.85 GB of RAM. As a disclaimer, I'll say that I'm also running the Chrome OS (I say OS because Chrome does all sorts of strange things to your computer. It might as well be an OS).

4GB RAM may be your problem. . .but I doubt it. As long as you've not got 10 browser windows open at the same time and subject to paging I think thrashing is more your issue. (Also DON'T turn off your page file. Please. I only mention that because you said your main rig's drives are "optimized".)

So thrashing:
http://www.sevenforums.com/performan...mpared-xp.html

I'm running Calibre stock, nothing but the default plugins.

Calibre uses betwen 125 & 163 MB of RAM on my system. Given that you've got 4GB of RAM, that probably shouldn't have much of an effect on your system (unless yours is using 3.85 GB just to run the OS more or less like mine in which case you may need to buy some RAM).

Running my defragmenter, music player, and scanning the drive for duplicate files, Calibre took 6 min 40 sec to copy my monster 303 book library. And this is on your standard 7200 RPM drive.

Copying the whole library took me 2 min 10 sec when I eliminated the thrashing.

Also, as BetterRed said above, maybe it's time to run CCleaner and decrapify your machine. I would suggest you run Malwarebytes, AdwCleaner, BleachBit (with all options checked, except for FREE DISK SPACE). Do not check Free Disk Space unless you want you leave your machine running this task overnight. Then reboot. Then defrag.

You'll have to sign into all your sites on your browser again, but small price to pay for a leaner, snappier system.

If Calibre is still running slow after all that, I'd nominate your AV as your culprit. It takes some AV's a little while to update their whitelists. Calibre 2.1 may not yet be on your AV's whitelist and so subjected to a bit of extra scrutiny and thus thrashing.