It's very doable actually. As a complete Google App Engine n00b + far from python wizardy I took me two weekends to hack the calibre code and to do the necessary plumbing.
I was interested in fetching books from my Google Drive and converting them (from EPUB, FB2) to MOBI so I can download them directly from the Kindle Web browser. YMMV. The following rough notes are from the diff of my hacks to stock calibre.
1) drop (rename) db, devices, gui2, trac, translations, web, most of the utils subdirectories
2) ebooks/oeb/transforms/rasterize.py - make it always raise exception
3) create python implementations plugins/magick.py (use PIL) and plugins/cPalmdoc.py (forward back to ebooks.compression.palmdoc.py_compress_doc)
4) some hacks in ptempfile.py; Very small changes in lock.py, constants.py and config_base.py
I used
http://code.google.com/p/pyfilesystem to create virtual filesystem.
Before importing calibre code:
1) set environment variables - "CALIBRE_WORKER_TEMP_DIR", "CALIBRE_CONFIG_DIRECTORY", set sys.resources_location and sys.extensions_location
2) create fake 'os' module, wire relevant methods (mkdir, chdir, stat, etc) to your code; wire fake_os.path.os and fake_os.path.genericpath.os back to fake_os. replace 'os' in sys.modules
3) replace builtin open with your code.
About 300 LOC total.
Then you can import calibre code; use in-memory file system for input, output files for calibre.
Once calibre finish the ebook conversion and created the output file, I copy it to blobstore so it can be downloaded.