I'm that way. On long driving trips while listening I've often realized that I have no idea how I got where I was. Not that I was lost. I just have no memory of driving the past hour.
At the same time I've never had any problems. Nobody honking at me or no near misses or anything like that. So I have to assume I did pretty well.
That was a kind of common thing to happen to me when driving and listening. However, in later years as I got older I began to find that if I talked to someone in the car with me I ceased being a safe driver. I'd get caught up in the conversation and forget to keep track of my driving. That's a very different experience for me than listening to a book.
By the way, I've driven places with my sister in the car with both of us listening to an audiobook and that was no problem, so it's not just that I wasn't aware of problems while listening. They didn't happen. Unless, of course, she was unaware of them as well.
Barry