I don't think it's an attention span problem, for those of you beating up on yourself for zoning out when listening to audio books.

Some people are audile learners and others are visual learners. I'm not surprised to hear that a lot of dedicated book readers lean more towards the visual spectrum!
So much can depend on the narrator, too. I really hated the voice acting for "The Forest of Hands and Teeth" that I listened to last year and the narrator just sounded so BORED.