I would say just a microfiber cloth if you want to be really safe.
The normal admonishment for not using Windex on displays is that many computer displays have antireflective coatings which can be stripped off by the chemicals in glass cleaners.
I'm not sure if the Reader has such coatings or not. Personally when I do use a cleaner, I suggest something like Sparkle, which doesn't have ammonia and I think does a better job of leaving a streak free clean then Windex anyway.
Anyway, I would expect if you want to use a liquid, a little dab of water or isopropyl alcohol on a soft cloth I would expect is ok.
I know this is crazy, but did the manual have anything to say on the subject (I didn't read it either)