Neil:
Your member info doesn't say where you are located, so I can't comment on your local laws. That said (and remembering that I am not a lawyer, but rather an interested semi-educated layperson), my understanding is that in US law
both the process and the outcome matter. In other words even when both start-state and end-state are perfectly legal you're still not OK if you use a not-legal path to get from one to the other.
Thus, if you own a printed book and scan it yourself (or have someone scan it for you) you've engaged in format shifting by way of a legal process.
<digression>Actually, we don't know for sure that this example is legal -- it's not directly enshrined in statute and hasn't been litigated to the best of my knowledge. But it's generally accepted as a reasonable analogue of format shifting cases that have been litigated in the music world. We'll go with that consensus for purposes of this discussion. </digression>
Conversely, if you own the paper book and download a copy that someone else has scanned both you and the uploader have violated the copyright owner's copyright. Your violation is minor, the uploader's violation might be major (depending on number of downloads).
In both scenarios the outcome is identical -- you have in your possession both the dead-tree-format book and the scans. But the first scenario got there via legal means, and the second scenario did not. It's worth noting, by the way, that in either scenario if you pass on ownership of the dead-tree format book (even by throwing it away!), you are obliged to delete or destroy the electronic copy! You don't have a
separate property right in the bits -- you got them as a consequence of having purchased the dead-tree-format copy. (Same same if you're format shifting music! Hold on to those CDs and LPs.)
One could argue many different ways in which this may or may not make sense... in fact we discuss it to death here at mobileread quite regularly. But this is the current state of US law, to the best of my understanding.
Xenophon
Obligatory disclaimer: I am not a lawyer and this is not legal advice. Should you require advice on which you may rely, please seek out a lawyer who specializes in the relevant aspect of the law.