The best formatting is here at Mobileread; the books are hand-crafted with consideration of the features of each file format and abilities of ebook devices in mind.
The selection is limited (for, um, internet values of "limited") to only a few thousand books (over 10k when you count all file formats) but excellent. And in many cases, the book designer is available for requests--if you really want a particular PD book without the pictures, they might be willing to provide that for you. And they
take requests.
Next best is probably feedbooks, most of which are Gutenberg productions with somewhat better formatting.
I wouldn't recommend ever paying for a public domain book from one of the mainstream publishers (unless you had some need for a particular version's commentary or annotations); they are often either using the Gutenberg edition as a base, or scanning-and-OCR'ing a paper copy, with less error correction than Gutenberg.