[Quoth]

Finally I got round to testing.
The proper prime marks, rather than italic ' and " giving ' and " do work on old mobi only Kindles. Not a problem on epubs or azw, of course. I've not tested a pre-epub Sony.
To enter the the real Prime and Double Prime characters on Windows if you have a separate number pad, then press Alt+8242 (press and hold the Alt key while you type 8242) for prime, or Alt+8243 for double prime. On MacOS either use the Character viewer or add the International Unicode Hex Keyboard and switch to it to enter 2032 for prime or 2033 for double prime in Unicode Hex. Windows and Linux have similar utilities.
In MS Word go Insert > Symbol. Enter 2032 for prime or 2033 for double prime in Unicode Hex.
In LO Writer, on any platform, go Insert > Special Character and put 2032 for ′ or 2033 for ″ in the Hexadecimal U box. Click on Add to Favourites and then the symbol.
The prime characters can only be inserted if they are actually in the character set. Actual symbols: ′ and ″. The relative weight will depend on the font family. They are a little bolder than italic ' and " in Georgia and quite light on the normal serif font on the Kindle DXG. I didn't try them in Courier, on the DXG, which does seem to render as a font.

I'm sure I've seen Sans on an unmodified DXG or older Kindle, but currently I've forgotten the recipe.