If you don't buy many books, perhaps it's not for you. However, been able to get books from the library and return them (or not) without late fees might be appealing. Samples of the first few chapters of a book remove the issue of buying without knowing if the purchase is going to be worth it (with the few exceptions of books that go downhil in the latter half)
For me though, the ereader removed the need to keep using up more and more storage space on books. It's not often I re-read a book, but having bought it I don't like throwing them out nor just passing them on, the ereader makes that a non-issue.
Since getting one I've found it more convenient to read with one hand and turn pages whilst holding a cuppa in the other, a minor and quite lazy plus I guess
I've only recently bought a kindle (replaced my hanlin, which in turn replaced a pda) and have to say the dictionary feature is a lot more useful than I originally thought it would be. Not sure what to make of popular highlights and notes, but I can see a future iteration of notes making for a great feature too.
Perhaps my favourite reason though is I don't like waiting for books to be delivered when I've decided to read something or having to go hunt down a copy. eBooks are instant gratification in that regard. Again, perhaps not a big plus for many, but for me, it's a huge pro.
Enough convenience features make it a great device for me and many others, but on the flip side there's going to be plenty of people for who the conveniences do not make up for the initial investment and until ebook prices sort themselves out in many cases you can be paying the same or more for an ebook as you could a paperback. So you're correct in saying they're not for everyone.