I just have two Calibre libraries: the main one and one I call Calibre Vault. All of the books in the main library are on my reader. I only move the books to the vault if I want to keep them, but don't think I will reread any time soon. Also, most of my non-fiction books are there.
I've toyed with the idea of doing more libraries, but I think the system I have now works well for me. It was a lot of trial and error to get to this point

Starting a new Calibre library seemed scary, but really it ended up being super easy. I have a custom column (for date archived) in the first position in that library so I know at a glance which library I am in. (This was added after I accidentally added a bunch of new books to the vault library).
I'm odd in that I only keep the final ePub in Calibre. I also have a digital library folder when I keep my original format and a copy of the final ePub. This folder has sub-folders of my favorite authors, a Miscellaneous folder for authors where I only have one of two of their books, and the Undetermined folder where I keep everything by new-to-me authors until I've read it and decided if I want to keep it or not.
I delete books I don't want to keep. I used to keep everything thinking "Well, maybe..." but if I didn't like it enough that I could possibly want to reread it someday it goes. (I make an exception for books within a series I otherwise enjoy).
My Calibre folder and Digital Library folder get backed up regularly, both to USB drives and DVD-Rom.
I was keeping my Digital Library folder in the cloud, but had an incident where some of my books up and disappeared on me. So I moved it out of the Dropbox folder. Now I maintain one of my backup copies there, but not the main version.
A friend of mine keeps a separate, empty Calibre library for all of her print books. Just the idea of setting one up for all of my print books exhausts me! I did a massive purge in my last move since I majorly downsized my house, but I still have several thousand print books. I still keep track of all the books I own, my TBR pile, and Want to Buy lists in various word documents as that's the way I have been doing so since I bought my first computer.
I think it all comes down to finding a system that works for you and that is easiest for you to maintain. Don't be afraid to experiment with it all. If adding & converting books as you buy doesn't work for you, maybe setting aside an hour or two on a regular basis once or twice a month would work.