What are "regular readers"? Are you assuming that they have the same level of technical knowledge/experience that you do??
What happens if a reader does see an error? Do they have the knowledge to realize that there are rendering errors with such and such version of Nook/kindle - shake their head and scoff at Nook/Kindle? Do they know that there are bugs in the conversion program that the publisher used - shake their head and scoff at the publisher? Or do they say to themselves "this is not a professional book - what author put out this garbage"?
In my HUMBLE opinion, I would not want to take the chance. I would want to provide the best possible document to the consumer. That means a clean, hand crafted, document. That's the advice Jon was giving on an earlier post.