Have a look at the
http://www.nicemice.net/par/
program
par is a paragraph reformatter that does remarkably clever job reformatting lines. It can recognize and properly wrap mails with multi level quotes, C comments, numbered lists, lists, bordered paragraphs, weirdly formatted text.
the [uncomplete] documentation is at
http://www.nicemice.net/par/par-doc.var
Par is Unix prorgam written in pure C, and I was able to compile it on Windows (using Borland free C compiler)
I am pretty sure you can borrow an idea or two by studying the code ;-)