Christopher,
you can use WiFi without automatically downloading the latest version of firmware but to do so requires patching your Touch.
The forum thread
Instructions for patching firmware 3.19.5761 contains all of the details. (Assuming you are using firmware 3.19.5761, else do a search for "patching firmware" and choose the version number that you wish to keep using.)