I agree that we can't think of and treat digital content the way we treat physical content. Digital content has more similarities to broadcast content, and the rules for use should be similar.
For instance, with a television program, we are free to watch it (of course), and we are free to record the program to watch later, possibly many times over. But we are not free to sell a videotape of that program that we recorded (in most markets).
However, there can be a used market for that TV program: Buying a prerecorded tape or DVD of the program--essentially the program, tied permanently to physical media--and reselling that physical media as a "used" product.
E-books should be considered in like vein. The digital file, as downloaded from the web, e-mailed, etc, is akin to a broadcast program that you are free to read, but not to re-record and resell. Anyone who would like to resell their content would need to purchase that content on permanent physical media (such as an SD card) and resell that.
The alternative to that would be the purchasing of a "license" that accompanied the digital file, which must be present in order to access the content, and which cannot be duplicated by consumers. When the license was resold, the original owner would be unable to access the file without the license.
Of course, for the license idea to work, there would need to be a method of identifying individuals, files, and the licenses they owned that granted them access. I will refrain from adding more, since it would be hard to be heard over the howls of privacy advocates...