I don't know about the ones you loaned out and didn't get back. I would be inclined to say you don't get to count them, just as I think you do get to count books you buy used or are given.
I can't promise it's legal to download copies of books you have, either. As I said, I'm not a lawyer. But it seems to me that it ought to be legal to scan them yourself under "fair use," so it ought to be legal to get a copy that someone else has scanned, if you already own the book in paper. (This is all according to my understanding of U.S. law, and may not apply in other countries.)
Note that I am not saying it's legal to scan and OCR a book and distribute it to the darknet without permission of the author/publisher/copyright holder. So even if it is legal to obtain a digital version, you may be relying on the illegal actions of someone else.
Imagine a hypothetical club which admits members who have purchased paper books, from wherever. Imagine further that this club has a way of verifying exactly which members have purchased which books. Now imagine that each member undertakes to scan and OCR a subset of their own books, and that club members coordinate so they don't duplicate efforts. Now imagine that the club allows members to share their digital copies only with those members who also own the same book in paper. I don't know how a U.S. court would rule on this sort of arrangement. It might matter to the court whether legal digital editions of each book existed. But I think it would be very hard to find a jury in the U.S. that would convict anyone in this club of any criminal wrongdoing if they were each able to show they had, in fact, paid for and still owned all the books they each held digital copies of.