Thanks for the info. So, I would need to install all of the libraries for Calibre (which in my case includes installing X.org) in order to just get the non-graphical elements to work. However, that would all primarily be a question of disk space, since I wouldn't have to actually run the parts of Calibre that use X and the other window rendering stuff.
I'll think about it. Maybe I'll just turn on my desktop a few minutes before I want to sync my ebook reader so that it has time to download.