I can only tell you about the ones I have first hand experience of, so I'll just say that the Sony 650 and PRS-T1 both satisfy those requirements (although I have no idea what the ogg music format is so you would have to check that).
No doubt others will be along shortly to suggest others. Whatever you get, I hope you enjoy it