There's a weird situation I can't nail down 100% where smartypants reverses a closing quote (makes it an opening one). When it does happen, it seems to be near an emdash entity (or character). But that seems to be bug, rather than a special typographic situation it doesn't handle.
I also thought of creating a user-editable list/dictionary of 'tis-type words that could be integrated into smarty (or another) script.
I think you've identified the two big "deal-breaker" scenarios where SmartyPants is concerned, though.