I use Calibre to manage my collection (about 250 books). And, speaking from my personal experience - it is a bit clumsy - but, I also don't open very often; therefore, minor inconvenience.
The database table could have less padding - to see more entries on the screen at one time. IMHO the interface could be re-worked entirely to give an improved User-experience.
Collection (playlist) building could be improved by matching similar organization software users are familiar with (most music players iTunes/WMP for example). Although I do like that I can highlight all the books - and set the collection (except order) in one fell swoop.
That it recognizes duplicate adds (downloaded books added to list) - but doesn't recognize duplicates when uploading to device - messed me up once (doubled up all my books on my device).
Having said that, coming from the WebApp world - when someone mentions "clunky", it's usually an interface issue - and, interface is usually the last thing to really come together nicely. (Windows and MacOS weren't always as easy/nice to figure out.) And, it is the best free app I've seen for the Mac (in fact, of the couple I tried - it was the only that worked straight from DL).
The app has a lot of potential - and I definitely appreciate Kovid's work to date. And look forward to the future.