@EvilWayz,
I was getting logged out of Ubuntu 11.04 at random times - nothing to do with calibre for me. Your issue may not be the same as mine of course, but there are certainly a lot of people complaining about random logouts.
I noticed yesterday that comments on one particular bug report for Ubuntu
774978 that a proposed patch had been issued for laptops in the synaptic touchpad code. I've enabled the proposed patches in my update manager and installed it. So far no random logouts - but being random means it might take a bit to be certain.

Hopefully it is the fix thats needed and will be released generally soon.