- Make sure "Allow apps downloaded from: App Store and identified developers" is selected in System Preferences > Security & Privacy > General
- Only download the Calibre update from https://calibre-ebook.com/download_osx
- Double click the downloaded calibre DMG and drag the icon into Apps. Select "replace existing"
- Profit
Or, explain the method you're using to update. I'm not sure what's going on there, sorry.