View Single Post
Old 10-11-2010, 07:20 PM   #41
chaley
Grand Sorcerer
chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.chaley ought to be getting tired of karma fortunes by now.
 
Posts: 12,471
Karma: 8025600
Join Date: Jan 2010
Location: Notts, England
Device: Kobo Libra 2
Quote:
Originally Posted by blastyblast View Post
(I'm sure I'll get some comments, like "those things are obvious", and they are to some degree, but hey at least I'm trying lol...).
Its obvious that you will get comments that its obvious that you will get comments that it is obvious that (stack overflow!)
Quote:
[...]Switching to a local drive cut initial load time in half, and saving changes by probably even more than that.
One of the things that Kovid and I discovered some time back while doing work on bulk edit was that SQLite creates a file per transaction to keep its journals. On Kovid's linux installation, this didn't seem to matter. On my Win7 installation, life was not good. We got a 2 order of magnitude improvement in bulk edit performance by grouping transactions. Evidence pointed toward rapid file creation/deletion being very expensive on NTFS (Windows journalling file system), and this can't help but be worse over a network.
Quote:
So I just started clicking through and removing books in formats that will never get used. That took quite a while, but seems to have paid off as well.
This one surprises me. The only thing in calibre's startup that looks at formats is building the tag browser format category, and that is a single query on a view. Calibre doesn't look at the actual files until you ask it to do something to one. Do you have an idea how much this cleanup helped?
Quote:
Three, I took Chaley's advice (by proxy) and defragged my server and local drives... Not kidding when I say it ran for 3 days non stop. (30 TB takes a while lol).
Do you have before and after times? The reason I ask is that suggesting defragging has occasionally met with derision. It would be nice to know what level of difference it made in your case.
Quote:
And four, I seem to have found some extra patience, lol. I love the app so much that I really don't care how long it takes to load, although Calibre is significantly faster now that I've cleaned house... It loads >5,000 books in less than 15 seconds. Nice.
Going from 120-180 seconds to 15 seconds is good work. It is hard not to like factors of 10.
Quote:
Thanks for the dialog folks, it helped.
Thank you for taking the time to test, and (even more) taking the time to report.
chaley is offline   Reply With Quote