I think performance should always be a concern. eReaders generally aren't really powerful machines. A tidy CSS is desirable, for sure, but you should verify the size of your HTML files too. If they are too big, performance will suffer a lot.
To achieve maximum performance, it helps to optimize your code with tools like
HTML Compressor,
CleanCSS, or
Helium.