With Calibre shut down (no tray icon, no content server):
1) Copy the entire Library
Folder. That is the books and metadata.
1b) Repeat for each
additional Library (if you have more than one)
2) (optional) Copy the Calibre configuration folder. This has your custom settings, plugins, preferences.

Kovid: Have you thought of making a small, standalone utility with a link on the Os startmenu, that would read the calibre configuration file, then copy all the
referenced folders to a user designated location?