I currently have over 20k titles in calibre, quite a lot of it being fanfic at the moment since I've not finished importing the more serious reading materials from their scattered locations. Of those already in Calibre's library, 12650 are currently on device and managed by Calibre Companion, which does an admirable job of sorting things.
I like having most of, if not my entire library with me, because I never know what I want to read at the moment: I might get a minute free and decide to switch to something else other than what I was reading halfway the last time. So I tend to have 5 or 6 books on the go, and my reading speed is generally in the 5-800+ words per minute range, thus I like having lots to read available that I can pick and choose from at random. I also enjoy rereading old favourites and want those with me at nearly all times.
My original plan was to stick the entire library onto the phone as soon as I finish importing and tidying up metadata for the non-fanfic stuff and wrangling the current jungle of tags. Sadly it seems that unless I find another device, probably a tablet or dedicated reader, certain problems from Android, my device and the ondevice database along with the extremely long sync period will make it so that 10000 is probably the optimal limit for the amount of ondevice files. And I'm nowhere near filling my 16GB SD card either.