Slow with large library

[BetterRed]

Originally Posted by luoto

About 14-20 seconds for a 25mb PDF (just tested one) and about 25 seconds for a 40mb ePuB. The status bar sits at zero percent for a time then rushes to 100 per cent and sits there for a second or two. All other file accesses on the system seem to be normal, e.g. opening Word documents, moving the files with Finder and the like. I guess I just must split the library until a move to a larger capacity SSD is possible and practical (the last time I looked multiple tb drives were very expensive).

That seems slow to me, but my knowledge of Macs is scant, especially on this sort of issue.

I did some similar tests - 16GB I7-7700, Windows 10 Pro 1903. The library I used has 180K items and it lives on a 4TB WD Caviar Black on a SATA III 6gb/s controller. I run 2 active content indexers (Windows and X1) and two active AVs (Windows Defender and Malwarebytes) - with no exclusions.

From a SanDisk NVMe SSD I added a 31MB PDF to the above library in 2.2 seconds, I don't have any large EPUBS so I added a 200MB CBZ, it took 4.6 seconds.

My experience is that putting libraries on an SSD rather than a decent SATA III controller/disk combo makes no discernible difference to calibre performance. Likewise with library size, the performance of my 180K item Main library is not significantly different to my 64 item Test library - which has dozens of custom columns, they're both PDQ.

BR

[theducks]

Originally Posted by luoto

About 14-20 seconds for a 25mb PDF (just tested one) and about 25 seconds for a 40mb ePuB. The status bar sits at zero percent for a time then rushes to 100 per cent and sits there for a second or two. All other file accesses on the system seem to be normal, e.g. opening Word documents, moving the files with Finder and the like. I guess I just must split the library until a move to a larger capacity SSD is possible and practical (the last time I looked multiple tb drives were very expensive).

ADDING a book involves COPING the book to the LIBRARY FOLDER (in addition to OPENING THE BOOK and extracting the metadata for the Library entry) . This is what is taking the time: The SIZE of the books file. A typical EPUB novel is under 1M, it happens fast

It has NOTHING to do with the number of ENTRIES already present. Yous system is I/O bound . Gaming systems are optimized for Graphics. You need a FILE SERVER design (Learned by Experience ) if you really can't wait

[BetterRed]

Originally Posted by luoto

Yes it is hidden. I just have the text menu on the left-hand side. 53k-ish books in the library.

Originally Posted by BetterRed

"text menu on the left-hand side" - not sure what you mean ? Can you post a screen shot

I still don't know what this 'text menu' is.

In the main book screen the only thing I can show in the left edge is the Tag Browser. I normally have it hidden and use a keyboard shortcut to Show and Hide it. The book details (where the cover, comments are shown) can be shown at the right or bottom edge, I normally have it at the bottom.

Originally Posted by luoto

About 14-20 seconds for a 25mb PDF (just tested one) and about 25 seconds for a 40mb ePuB. The status bar sits at zero percent for a time then rushes to 100 per cent and sits there for a second or two. All other file accesses on the system seem to be normal, e.g. opening Word documents, moving the files with Finder and the like. I guess I just must split the library until a move to a larger capacity SSD is possible and practical (the last time I looked multiple tb drives were very expensive).

How long does it take to copy the PDF from where it is into a scratch folder on the 'library' drive - ditto the EPUB.

The PDF and CBZ files I used copied from C:\_scratch (the NVMe SSD) to D:\_scratch (the SATA III HDD) in 2.1 secs and 4.1 secs - which are fractionally faster than the calibre add times. But in addition to copying the files, the calibre add also creates a book folder, updates its database, extracts and writes the cover.jpgs, and creates the book metadata.opf files.

If the copy times are substantially shorter than the calibre add times (e.g. 2-5 seconds rather 15-25 seconds) that suggests something is interrupting calibre's processing - like a virus scanner, content indexer, encrypter etc. Try killing all extraneous tasks.

Also try adding items to a newly created empty library, if the times are significantly shorter then its something related to the existing library. The Job Spy plugin has a tool to list custom columns to a CSV, if you post it we can have a look for any 'oddities'.


Last resort: install Windows under Boot Camp on your Mac

BR