It is an interesting question though, isn't it. How does one start building an ebook library? This is what I have been recommending to my friends who have recently purchased eReaders:
1. Decide on a core format
The two main formats at the moment are ePub and Mobi. If you intend to get a Kindle, choose mobi as your default format, other than that pick ePub.
2. Decide how you want to store and organise your ebooks
It may not seem like a big deal now, when you have one or two ebooks downloaded onto your desktop, but you'll be surprised at how fast you get confused about what you already have! And if you damage your eReader or choose to upgrade, you'll want to be able to move them to your new device easily. So, you can leave them "in the cloud" with Amazon, let the Adobe ADE software catalog them, keep back up copies on an USB Flash Drive... personally, I catalog my ebooks with Calibre and back them up on Dropbox.
3. Explore the Public Domain ebooks at MobileRead, and download the ones you are interested in.
There's no point in paying $5 for a badly formatted, DRM-encrypted Public Domain work. And while there are a number of sites making PD work available for free, MobileRead versions are almost always the best.
4. Poke about the internet for (legal) freebies
Epubbooks.com, Feedbooks, Manybooks have more public domain ebooks, and some ebooks placed there by the authors themselves. And pop along to your favourite authors' sites. It's surprising how many of them have made freebies available.
5. Look for authors you like at ebookstores that sells DRM-free ebooks.
The advantage of checking places like Smashwords, Fictionwise (multiformat), Baen, Samhain etc. is that not only do you not have to bother making a decision about To-Strip-Or-Not-To-Strip, but most of these sites a) store your purchases on virtual bookshelf and b) make them available to in multiple formats. So if a few years down the line you smack yourself on the forehead and realise you actually prefer a different format, if the site is still up and running you can just bulk download your previous purchases in your new prefered format.
6. NOW take a look at the Big Stores
If you want to, I mean. 'cos your ebook collection is going to be pretty massive at this point
As for your specific queries
For me reading ebooks must have:
- the ability to look up words with a built in dictionary
The dictionary built into the ebook or the reading device? Because I know both Kindle and Sony Readers have dictionary support, but AFAIK the current ebook formats don't allow for attached dictionary files.
- takes notes/annotations, highlights etc...
again, that's more a hardware query
- be able to read those notes etc... while reading ebooks on different devices.
As in, if you switch from reading your ebook on, say, a Sony Reader to an Iliad? AFAIK this is not possible as the format in which notes are stored is not standardised from one manufacturing company to another.
- be able to purchase ebooks from any ebook store and use it on whatever device I'm using and have the above features.
You, me, and everybody on this site wants this *sigh*