That doesnt have anything to do with calibre, you will need to ask someone familiar with whatever set of layers are complicating your life.
If I were you, I would simply install the official calibre binaries and run them without all this complication. Pass the path to an SSL certificate to the server, turn on basic auth and be on your merry way. Or if you want to integrate it behind another main server there are guides for apache and nginx, even in the calibre manual
https://manual.calibre-ebook.com/ser...-other-servers