Okay, I think I made some progress on the problem here after some Googling...
Bear with me here, but I stumbled upon this barely coherent post:
https://topic.alibabacloud.com/a/kin..._31182897.html
The important sentence that caught my eye was "After extracting the Epub file, the ID of the link to the document exists, but the ID value 0-C6789AC79B5C427B831DE25D238BF6FA value corresponding to the tag is part0000.html <body> The body needs to be modified to a DIV tag kindlegen only to know, while the extra layer of a <body>"
With my limited knowledge of this stuff, for every link referenced in the ToC (for the book I was testing with), the <body> html tag had an "id=xxx". If I changed all of these <body> tags to <div> tags in Sigil, then Signal would offer to automatically repair the HTML files to add back body tags. My HTML files would start like this:
...
<body id="xxx" class="calibre">
...
</body>
and end up like this:
...
<body>
<div id="xxx" class="calibre">
...
</div>
</body>
This fixed things! I am not sure if I understand why exactly, but I guess the Kindle converter to epub doesn't like the ID in the body tag?
Anyway, upon further testing, I did find some AZW3->EPUB conversions that did not have this problem, so it isn't necessary an issue with the conversion. Sorry about that! I assume there isn't some conversion setting in Calibre to automatically fix this very specific issue I discovered, yeah?