I have always admired women who carry small purses. I don't know how anyone can get by with just the items you've listed above. My wallet alone would take up the whole 6x4 dimensions you quoted.
I look in my purse and don't think it's full of stuff, but certainly more stuff than you carry!
My best suggestion would be to check out something like Lesportsac. You can get them at almost any department store, and they come in the most amazing variety of sizes. Plus they are light as a feather and won't add any weight to the stuff you're carrying. (Not that it sounds like you have that problem.)
http://www.lesportsac.com/
And WHAT are you doing to your purses that they wear out so fast?