Yes, they respect robots.txt, as well as the no-cache directive. I shouldn't have to put a sign on my door that says "please don't enter my house" or explicitly tag all my belongings "don't steal". My sites' contents are already protected by copyright. Google ignores that. They've already been sued, but the clueless judge ruled they operated under fair use and added value to the content.
Let book authors meditate on that.
Historical Link, and an excerpt:
"Google, for its part, says the project benefits authors and publishers by raising awareness of their books..."
Hmmm, haven't I heard that argument before? Oh yes, from people here claiming illegal pirated copies of books are good because it makes authors more visible/prominent, increasing the chances that people might eventually make legitimate purchases.