You know what? I strongly suspect that the slowness is due to my external hard drive, a Western Digital My Passport, being formatted as NTFS and being accessed through a third-party driver.
This benchmark shows that such drivers are FAR slower than HFS+ in dealing with folders with lots of files and subfolders.
So that may be the difference right there.