On this site (mobileread) just click the E-books link in the blue bar at the top of the page and select Mobipocket. This is the file type that is compatible with the Kindle. I believe Mobileread recently just passed the 14,000 files mark, so that is a lot of public domain books from which to choose. (Oh yeah, you can also use 1001books.com to easily find and download only the most popular public domain books.)
If you want free commercial ebooks for your Kindle then go
HERE, where you can download newly released books where the publishers are giving away books to suck you into a series, introduce an author, or for some other marketing reason.