hi @GUdrus, i'm a medical student AND doing a phd, so i have to read both medical textbooks and scientific pdfs.
at the moment i have a tablet PC (Dell Latitude XT) with Bluebeam PDF Revu to read PDFs, OneNote for storing notes, Mindjet MindManager for taking short-term notes. (these are all tablet-friendly programs) unfortunately most proper tablet computers that i know don't have very high specs, so you'll find that it's not productive to be running more than 3 programs at a time.
it's OK to read books and journal articles on a tablet PC, but it depends on whether your eyes can take the strain. can you manage with long hours of reading medical / academic text on your current computer screen? if you can, then a tablet PC (proper laptop, with a decent processor) will fit your needs.
don't expect wonders from a touch screen. i personally like big screens so that i can view a whole page without having to pan and zoom. imho touch technology at its current state isn't precise enough, and winds up being rather annoying.
also, i'm not sure how much colour you need - usually a lot of diagrams and pictures are passable in black and white, if you really do need to view something in colour you can always go check it on a proper computer. i say this because my girlfriend is happy reading her medical textbooks on a Kindle 3, which has only greyscale, and she doesn't mind pan and zoom.
you're very sensible to not prioritise annotations. from personal experience, it's more useful to take notes in a separate location, where you can quickly browse through them, rather than have to flip through the book page by page.
however, my eyes tire quite easily and i don't like to read for a long period of time on my tablet PC screen. so i've pre-ordered a pocketbook 903 because it's one of the few 9.7" e-ink readers that supports a large variety of formats. i decided against the Kindle DXG because it only handles PDF natively, and even so, i've heard that the DXG pdf renderer (? if that is the correct term) isn't as good as it is on the Kindle 3s. that said, i am still waiting for my PB 903 to arrive.....