CALM Consolidation Speed Versus Calibre Speed Copying 50,000 Books
@Tanjamuse:
As a benchmark, for the purposes of discussion only, assume that for Calibre to copy all of your 50,000 books from Library 1 to Library 2 would take ~ 25 hours. I have personally experienced that rate of copy when I used a HD and not a SSD.
A "real" copy not only copies the metadata, but also all of the formats and covers and metadata.opf files, and then creates the necessary directory structure. Very RAM and disk and CPU intensive.
CALM copies only the metadata. No files. No directory structure. No formats. No covers. Just the metadata from the Source Library metadata.db to the CALM Target Library metadata.db. However, a metadata.db is a physical file that has to be physically updated with a physical temporary journal file. CALM uses a lot of RAM while copying the metadata via SQLite. So, CALM too is RAM and disk and CPU intensive. Just not as much as Calibre is for the reasons mentioned above. The CALM Job Log tells you how much RAM it used per Library, and the maximum RAM used at any point within the Consolidation Job.
So, CALM will copy (as a part of Consolidation) those same metadata-only 50,000 "books" much faster than Calibre would do when executing a "real" copy.
How much faster? Depends. Certainly no faster than 10% of the 25 hours (for the purposes of discussion only) that it would take Calibre, which would equal 2.5 hours. If it takes 20% of the 25 Calibre hours, then it would take 5 hours.
You get the picture.
When I upgraded my HD to a SSD (Solid State Drive), CALM (and everything else) ran much faster.
DaltonST
|