For the vast majority of mainstream titles, I am now seeing IDENTICAL pricing on Amazon, Borders, Kobo, and the Sony Store, so please cite some specific examples, so folks can decide if the titles you are talking about are something that would effect their own interests.
Before buying my reader I checked Amazon and Kobo on about 50 random titles from both New York Times best seller list, as well as several of my long time favorite authors. EVERY ONE WAS THE SAME PRICE. Also, for the authors that I am interested in, AMAZON'S LIST OF AVAILABLE TITLES WAS NO BETTER THAN KOBO.
As far as library services go - If you have a friend or relative that does live in an area where the library has 'overdrive' Adobe EPUB books available then you can look into getting a library card there using that address. For electronic checkout, access to the overdrive service is via the web, so after you have an account, it doesn't really matter where you access it from.
In regard to public domain material, please check out the public domain section of
www.feedbooks.com I have found that feedbooks not only has better proofed and formatted EPUB titles, but also is has a much better selection (thousands of titles) from more recent 'copyrights lapsed' works from the fifties and even sixties (so it's not all "pre great depression").
And lastly, YES it is not too difficult to 'unswindle' the Amazon books so you can convert them to EPUB using Calibre, but personally I avoid this option, not because I think it's immoral (after all it's YOUR book after you buy it right?), but rather because, contrary to your experience, I find that I CAN EASILY FIND THE SAME TITLES ELSEWHERE, AND CHOOSE NOT TO DO BUSINESS WITH AMAZON. If Amazon doesn't want to support open EPUB titles that can be read on ANY reader, then they don't want my business, AND THAT'S JUST FINE WITH ME.