I think you'll find that a lot of people who read for pure pleasure (myself included) rarely worry about retention (even though I believe I retain a good deal of what I read). And as far as comprehension goes ... well that depends on a huge number of factors to begin with. One of the main ones is the material being read. Another is "why am I reading this?"
I'm not reading non-fiction and I'm not reading for study or class, so if I need to slow my reading way down in order to fully comprehend what's written, then: a) the author is doing a bad job, or b) the author's style just isn't for me.
At this point in my life, I'm rarely reading to flex my mental muscle. I did all my strenuous reading years ago. So if I can't cruise through a novel, I'm probably going to dump it for one that I can (unless something intangible keeps tugging me back). That's not to say I have to love the material, but the words need to flow. If I have to constantly slow down or back up and re-read for full comprehension, then I'm not likely to finish that work. That kind of reading isn't enjoyable for me.
I feel no obligation to "stretch myself" any more when it comes to reading. I already stretch myself pretty thin with a very eclectic mix of fictional subject matter/genre, so I feel no real obligation to read purposefully difficult prose in order to grow as a reader.
In my mind, readers who read a seemingly incredible amount of material (over the course of a year) do so because they know what they like. And because of that, they're familiar enough with what they like that they're selecting works that they won't have any problems comprehending at their particular reading speed. Happy/engrossed readers read a
lot (fast or slow). And in my opinion, most fiction readers aren't really that concerned with retention to begin with. They'll just read it again if they forget stuff (and they liked it well enough the first time).
I learned long ago that there's very few concrete and universal conclusions one can draw about reading speed, comprehension, retention, intelligence, etc ... People just do the same things differently. Not better, not worse, just differently.
I go through phases, myself. Some months I'll consume books at a terrific pace, some months I'll binge-watch some TV series (and I feel equally rewarded by both experiences). I'm about to go into baseball and gardening mode, so my reading-time will drop considerably. I won't feel the least bit guilty about it either--I love baseball and gardening!
Much like my wide variety of fictional reading interests, I also have a wide variety of hobbies. I've never really felt compelled to pick an all time favorite. So I will continue to do them all (each to the detriment of another).