[NiLuJe]

There's indeed only so much you can do to paper over the fact that FAT32 should just die a fiery death ;o).

And, ideally for this use-case, be replaced with a filesystem that makes sense (i.e., is actually journaled, not like you, exFAT ;p).

TL;DR: This is (unfortunately) why stuff moved away from USBMS: the whole thing is a pain to integrate into a system properly in the first place, and it usually sits on top of a truly terrible filesystem, putting it in the very situation it's the absolute worst at handling safely :/.