Thanks kiwikobo, I will try Sigil.
I also did another experiment on chapter splitting by setting the chapter split threshold very high. This way the book is still split into chapters, but the chapters are not split into parts.
The result was significantly faster load time, 1.5 minutes, down from 2 minutes. I think this indicates that a significant amount of time is spent gluing back together chapters that have been split into parts. If the Darwin book mentioned above is all one chapter, maybe the huge chapter is being glued back together on load.
Truly separate smaller chapters look like a good approach.