I've found that some books are far more rewarding in re-reading than others. This can be especially obvious in series with plot arcs that cover the whole series, or with continuous character development - I've discovered all sorts of little clues and things upon re-reading that seemed like throwaway lines or mentions upon the first read, but then after a plot twist or reveal in the fifth or sixth book in the series, you suddenly realise that the clues were there all along.
Other books though are pretty straightforward - what you get out of it on the first read is the same what you get out of it when re-reading, apart from the element of surprise (if re-reading fairly soon after the first read). So it really does depend for me, and it's one factor that determines whether I can happily re-read a series 4-5 times or whether once will be enough.
Of course, I also don't have the best memory around, and I read a fair bit, so chances are that unless a book has been *really* memorable for some reason (which happens with maybe 1-2 books a year, and only if I can talk about it with someone), if I pick it up again 3-4 years later, I won't remember much of the plot, never mind anything else.
It also depends on the genre - with straightforward mysteries, I wouldn't re-read them any time soon after the first read (I would many years later though, once I'm sure I've forgotten who did what!), while with other books, it really is the journey that matters, more than whether I remember the plot outcome/details or not. I think for me this is largely a difference between plot-driven books and character-driven books, too.