Assuming it fits into your pockets and otherwise fulfills your needs (battery life, OS, apps, connectivity, etc) I don't know why you wouldn't want a bigger screen. I suppose if you do a lot of talking on the phone and have small hands it could be annoying to hold a really large phone for a long time.
I recently bought the Nexus 4. I have a pair of jeans in which I cannot comfortably sit down when the N4 is in my front pocket. I just wear my other jeans.