Actually this issue is a big problem. Considering that DE or something like it has to run on an embedded device, the performance penalty is going to be a major issue. That's one of the good things about LRF. Because it's so simple, a renderer that performs reasonably in a embedded context can be written.
Splitting a book into multiple files wont help if you want to have the number of pages in the book displayed, as for that it would have to parse all the files anyway. Ofcourse, if you feel the reader can do without that then the rendering in the existing DE will be fast enough as it wont have to do pre layouting.
The other partial solution is to pre-cache the layout information like the SONY Reader does. But the problem with that is you wont give the user full control over font sizes, since you can only pre-cache for a limited number of font sizes.