It appears to me that all of the discussion bearing on the ethics of copying are couched in terms of the creator's right to make money off of the sale of the book.
It seems to me that if a book has been allowed to go out of print, it amounts to an abandonment the right to make a profit. Now, there could be some quibbling about this point, as when the author still wants to sell the book but can't find anyone who will publish it. But on the whole, I think it is reasonable to suppose that a book goes out of print because there's no longer any profit to be made.
So if a book is out of print, I have no problem with copying it in any fashion, unless there is some other right that is being violated. And I don't see that any other moral right has been advanced for out of print books.
But if the book is still in print, there's obviously a continuing intent by the author/publisher to profit from sales, and it is unethical to, in essence, steal that right to profit. So it appears to me that making a copy of the book is ethical only if you have personally bought a copy, which respects that right.
From this perspective, it seems to me that making a copy of a library book, where the book is still in print and you can buy one yourself, is not ethical. Basically, when you copy the library book, you are not copying a book you have bought. You are copying a book someone else bought. You have no more ethical right to do that than you have to copy your friend's purchased copy.
Making a copy of a library book is not time-shifting, although it might seem to be so superficially. Timeshifting involves your right to make a copy of something that the creator has sold you, even if he has sold it to you for free, as with a TV program. But the creator has not sold you the library book. He has sold the book to the library. Even assuming that in the sales agreement, the creator did not restrict the right to copy, it is only the library which has the right to timeshift.
So in my view, you can ethically copy a book which is no longer in print, but you can't copy one that is in print unless you buy your own copy.