I've never been much of one for expandable storage. The average book size is 500 KB, and with an average internal storage of 3.5 GB, you can store thousands of books at a time.
Of course, this does not take into account people who read e.g. manga.
As AnemicOak said regarding format support, calibre makes formats something you don't have to think about -- and you are anyway going to sideload non-DRMed ebooks, right?
I cannot think of any devices now made without WiFi capabilities, but you don't have to turn the WiFi on if you don't want to, so we can safely ignore that factor.
Touchscreens are also pretty standard, so you will have to get an older device if you specifically want to avoid them. Would the Kindle Voyage/Oasis page turn buttons work instead?