Yeah getting calibre running on linux can be a pain, I tend to use cutting edge library versions. Have you tried the binary installer?
As for MOBI files that don't work with the Cybook, open a ticket and attach the source files. We usually fix bugs in a matter of days.
The LIT support isn't as developed at the moment, which is why it is not included in the GUI.