I find it fascinating how differently we all process things.
Like the OP, I have do do something else while listening to a book or lecture, or my mind wanders. When I was in school, I took a lot of notes (and did a lot of doodling) in lecture classes.
However, unlike the OP, I can easily concentrate on reading written material.
I can't imagine I would ever be well-coordinated enough to learn to knit or crochet by itself, much less do it while reading. Gotta admire those who can!