If you're enabling preprocessing and it's not working you could open a bug with the book, it should work with those types of chapter headings and preprocessing.
For the dedication it's probably easier for you to use Sigil after Calibre has converted from rtf to epub. I don't think there is a way to do this using rtf. It can be done if you use txt and markup, but that requires a few more technical hoops to jump through.
If you get chapter detection working then Calibre normally splits each chapter into it's own file.