You know, it may be - for at least some of them - it's going to be pretty subjective. I happen to know that a couple of the narrators that drive me crazy are favorites of at least one of the regular posters over on the Free/Bargain Romance thread.
I actually recently posted a
bad narrator review on Audible's page for the book
The Hot Zone by Jayne Castle. I listed to at least part of another book (
The Nazi Officer's Wife) by the same narrator and for that book she was fine, but she butchered this one! To me, she sounded like she was doing a William Shatner impression. On top of that, she sounds to be an older woman who probably smokes (I can picture her with a rocks glass and a cigarette) - and the main character is in her 20s. It's just too, too distracting.