If you can't even click tick boxes then it sounds to me like something is badly wrong with your installation. I would uninstall it, download the latest version again and re-install it, then see what happens.
I have been using Calibre for some time now, and whilst it can be slow to start and sometimes a little slow to respond, it has not been a problem.
One thing to mention is that you don't tend to see any immediate feedback when the conversion is complete, as it does the conversion in a background process. After you have clicked OK in the conversion dialog take a look in the bottom right. If it says "Jobs: 1" and the little whirly thing is whirling (
) then all is OK. Wait until it finishes whirling and says "Jobs: 0" and then take a look you should find you now have the old lit format and the new mobi one. It can take a while depending on the size of book and speed of computer but mine (an old laptop) generally completes within a minute.