The "arrow" is a linebreak rather than a paragraph break. Assuming the "arrows" are only in the places where you would expect to see an end-of-paragraph, if you load the RTF into Word you can do a Find-and-ReplaceAll on the whole document.
Find: ^l (that's a lowercase L not a One)
Replace: ^p
|