I've ended up converting the flourishes I want to SVG and displaying them as images.
I've had trouble getting some dingbats to display at all, and also I like the flourish to be a fixed size relative to the screen width. I know how to do that with images, but not with text. Finally, I can set the <IMG> alt-attribute to something like -->-*-<--, so there's an OK fallback, instead of spurious letters popping up.