The difference between the portable and regular versions of calibre are that the portable one has calibre-portable.exe -- which stores the config and libraries in the same folder -- and the fact that you don't need administrator rights to extract the Calibre/ folder with the binaries.
Personally, I would use the
PortableApps.com version of calibre which handles the configuration using the standard PA.com Format, and also allows you to store libraries anywhere you want (their launcher rewrites the paths to deal with changing Drive letters).
Not knocking Kovid's portable installer, but the PA.com guys have been doing this for a lot longer, and they specialize.
Kovid's version just hardcodes some values, like the original batch script except not editable.