@victorhooi: TL;DR: Because reasons ;p. You can read Calibre's code for more details, or try to pull your hair out by digging up contradictory would-be explanations on this very forum (;p), although that will only shed light on the how, not the why, which can probably be summed up by my initial answer ;p.
But no matter the reason, it basically boils down to the fact that, no, you really can't work it around without changing your workflow, as @HarryT pointed out. On the plus side, you're not doing anything wrong per se

.
There are also a few different CLI tools that handle basic Kindle library maintenance if for some reason you can't stand Calibre. Check the Kindle Dev subforum and/or GH. The most recent one that comes to mind is Librarian, but I seem to remember one of the KCC guys doing something of the style, maybe?