Yup, for sure, there are serious performance differences between the Vox and Touch. It's not a good or bad thing, just different. Downloading happens in the background so if it's 30 seconds or 3 minutes makes no difference (to me, anyway, since I am likely doing something else).
I like the greater text control of the Touch, the great battery life, lightweight, and the way it cradles into one hand.
But I really like the Vox for the ability to do other stuff, as well as access my Kobo books -- like direct over the air access to the public library and the ability to do surfing for news and play a few time wasting games (like Sudoku).
Having both also means my husband and I can be reading the same thing at the same time ... hard to do that with a paperback!