Switching to a previously processed cover is pretty fast, it just has to recognize the ASIN of the current book and use the cached cover.
But extracting a cover for the first time is slower. Usually it doesn't matter, because you are reading the book and it happens behind the scenes
but you will notice if you open a new book and then put the Kindle to sleep within ~10-15 seconds it will be stuck on the old cover.
EDIT: I guess I was (partially) wrong? See NiLuJe's authoritative answer below...