Portable is working as designed. The Library(s) are expected in the folder.
You can always switch to other folders (anywhere) BUT, only those in the default location, will be remembered upon the next launch.
If tou reserve a drive letter for you USB stick, Calibre (the Win version), can reopen the library that is on the USB stick