Exactly. When a physical book is sold to a used book store, given away etc., nothing is being copied. A copy is being transferred, and the original owner no longer has one.
With digital downloads, piracy etc. copyrighted material is being copied and illegally distributed etc.
And downloading a digital copy of something you didn't pay for is the closest you can get to theft for digital product since there's no way to shoplift it etc.