I keep around 20 books on each device. When I've finished a book I usually delete it (if possible). Apart from my next few "planned" reads I keep a selection of non-fiction and short story collections of different kinds on the device so there's always something to fit the moment at hand.
If I had hundreds of books on my device there's a risk I'd just spend my time going through the library over and over trying to decide, instead of actually reading. I know myself well enough to avoid at least some of the things I deal badly with...