Refreshing does clean up the text. By the time you get to the 5th page (with serif fonts its easier to see) you get lots of jaggies and ghosting from previous words and your letters look too faint.
I manually refresh every page (double tap "N" button) and notice the difference -- especially when in direct sunlight.
And I do use alternate fonts. If you edit your epub file you can embed whatever font you want!
