SSDs are great

[ilovejedd]

Yeah, weird title for this section but just wanted to comment how awesome SSDs are for large conversion jobs. I have a PHP script (max execution time=10 mins) that calls ebook-convert command-line and it kept timing out on a 50 chapter, 4MB HTML file. I wasn't getting any time-out errors when I first converted it 3 days ago (with a 5 minute time limit, even) so I was scratching my head on why I'm suddenly getting this issue. I forgot I was running the virtual disk from a mechanical drive since I ran out of space on the virtual machines SSD. Made space on my primary SSD so I could move the virtual machine to it and the conversion finished without a hitch and with plenty of time to spare.

The file was actually part of a 2,798 HTML -> ePub batch conversion queue. First run was on an Intel X25-V 40GB SSD and the whole job took less than 5 hours to complete (queued it at midnight and the job was finished when I woke up at 5AM). Made some changes to the PHP script (nothing that would greatly affect conversion to ePub) so I re-ran the script, this time on a Seagate Barracuda LP 2TB and there were still 500 or so files not converted after running for 6.5 hours. I reckon I'd be running the conversion again, this time on an Intel X25-M 120GB. It'll be interesting to see if there will be any difference in performance between the X25-M 120GB and the X25-V 40GB.

In any case, for folks working with large libraries, do yourself a favor and use an SSD for your Calibre library. Even an inexpensive 40~60GB SSD does wonders. Here I was thinking maybe Calibre might require a faster CPU or more cores when really, disk throughput and increased I/O apparently makes a more noticeable performance improvement.

[DoctorOhh]

Originally Posted by ilovejedd

Here I was thinking maybe Calibre might require a faster CPU or more cores when really, disk throughput and increased I/O apparently makes a more noticeable performance improvement.

I can't speak to your experiment and have little doubt that a SSD would speed things up, but I recently converted 9000 ePubs on my I7 computer and I averaged just over 2000 books an hour. It was sweet seeing all 8 cores churning away converting those books. The machine also has a SATA III 6gps drive. Even so there were a couple of books that took longer than 10 minutes, I don't have a terminate time limit set.

[ilovejedd]

Originally Posted by dwanthny

I can't speak to your experiment and have little doubt that a SSD would speed things up, but I recently converted 9000 ePubs on my I7 computer and I averaged just over 2000 books an hour. It was sweet seeing all 8 cores churning away converting those books. The machine also has a SATA III 6gps drive. Even so there were a couple of books that took longer than 10 minutes, I don't have a terminate time limit set.

Is that using the GUI? On the SSD, I was getting a bit over 550 books (files ranged in size from 5KB to 5MB) per hour on average on a single core (only had 1 core assigned per VM). One thing I've noticed is on the SSD, CPU utilization was 100% all the time. On the mechanical drive, CPU usage was mostly 100% but it occasionally dipped to 50% or below. I can only imagine that particular problem will be exacerbated when running multiple instances of ebook-convert. In fairness, while a bit aged, the CPU I was using was no slouch (Core i7-860). Still, I'm chomping at the bit to upgrade to Ivy Bridge.

[DoctorOhh]

Originally Posted by ilovejedd

Is that using the GUI? On the SSD, I was getting a bit over 550 books (files ranged in size from 5KB to 5MB) per hour on average on a single core (only had 1 core assigned per VM). One thing I've noticed is on the SSD, CPU utilization was 100% all the time. On the mechanical drive, CPU usage was mostly 100% but it occasionally dipped to 50% or below. I can only imagine that particular problem will be exacerbated when running multiple instances of ebook-convert. In fairness, while a bit aged, the CPU I was using was no slouch

Good info, thanks!