I believe the story your talking about is David Eddings' The Losers. I don't remember the sexual affair with the aunt but your overall description sounds spot on with it. Definitely a great read too (although this may be colored as it's been more than 5 or so years since I've read it).
The Losers - David Eddings