The Sony is thicker (and colder) than the Kindle to hold. I wouldn't want to read the Sony without its cover (especially in our bedroom!). The Kindle has a nice feeling.
The Sony is smaller though, due to the lack of keyboard. If you like to write whole novels on it, I'd not get the Sony, but the Kindle. If you prefer a smaller device, get the Sony.
I love the touchscreen of the Sony. Because it's not a touch
screen, you don't have to touch the screen, it's very light (how much force does it take to swipe the air?). And you can use every single object you want (though, I'd keep sharp tipped object away from the screen

)
Disadvantage of the Sony: it won't activate the dictionary when I'm reading my home-generated PDF's. I haven't tried it on the Kindle yet (I'll try tonight).
The memory card is nice but the Kindle has a lot of internal memory. But I do like the ability to switch cards.
Formats, I don't care about, I reformat all my books to PDF anyway. Made up specifically for a 5" screen (so it'll look great on a 6" as well), so no need to reflow or anything of that kind. And the same source creates epubs and mobipocket books as well, so any device would do for me.