Hi LordFluffy! Welcome to MobileRead!
You can get books anywhere that has Adobe epub or pdf titles. There are many sources listed in the MobileRead Wiki at the very top of the page.
One really good source is public libraries, most of which now carry ebooks you can download either to your computer, using Adobe Digital Editions, then to your PE using the USB cable, or directly to your eDGe into Overdrive Media Console.
I buy books from Booksonboard.com, kobo.com, the Sony ebookstore (download to their software on my computer), and Amazon.com. The Amazon books are readable in the Kindle Reader on the LCD side only (they are in a proprietary format).
You can use a program called Calibre (do you know it?) on your computer to turn all kinds of documents into epub or pdf which you can then send to your PE through the USB cable or over a LAN.
Most of the ebookstores have some free books, and there is a large collection of free books here on MobileRead. Take a look under E-Books at the top of this page.
I hope you will find many good books to read on your new PE!
P.S. If you need clarification, please ask! I love reading on my PE and especially on my EE!