One thing some people do that helps is to carry a few different SD cards with different books on them, then they swap out the SD cards depending on which book they need to access. To make this feasible though, you'd still need to cut down from 12,000+ pdfs.
I currently have about 275 books--most on my internal memory and about five on my SD card. When I make changes or have to reboot my reader it probably takes about three or four minutes to process those files. (I've never actually timed it.)
Good luck with your organization!
dordale