Closing the Android File Transfer application does not shut down the AFT agent. If you plugin in an MTP device after shutting down the application, the agent will try to launch the application again, and that might be interfering with calibre when it checks for connected devices.
I stopped using AFT because you cannot prevent it from installing the agent as a Login item and I just want to have control over what connects to MTP and when, without having to constantly shut down AFT.
Instead I use OpenMTP. It has an option to install a 'watcher' but otherwise when you quit the app, it's gone.
Interestingly the download link is gone from
https://www.android.com/filetransfer/
I was able to use
brew install (so I could re-check the behavior of AFT), but I don't know if that really works now, because it found the AFT dmg in my brew cache and never tried the official download URL.
At any rate my advice would be to uninstall AFT and try calibre again. It's working for me. And then get OpenMTP. (brew install --cask openmtp, or from GitHub)
FWIW I can reproduce the exact error you are seeing by waiting until calibre starts to connect and tapping the Disconnect button on Scribe.