For me it's about the lack of refresh visual noise. If you've ever taken a picture of a LCD display you see what I'm talking about. Somehow all this noise is filtered out somewhere between my optic nerves and (few remaining) brain cells into a stable image. My theory is that it's this noise filtering that generates the fatigue and it's difficult to measure and impacts people differently.
If you compare a progressive scan HDTV image to an interlaced image at the same resolution it's a similar comparison but the e-ink has no refresh. Most people would find the progressive scan is better to look at.
Some LCD displays do a much better job of filtering the noise then others. I haven't seen a Nokia N800 to compare but it sounds like they've done a good job with the display.