There are two versions of the 4Gb Memory Stick Pro Duo - Normal and High Speed. The normal one works, the High Speed one doesn't.
Are you sure your Memory Stick is not the High Speed one with the red stripe?
Check this page:-
http://esupport.sony.com/EN/info/mspro-reader.html
Other thing - have you got the latest version of the firmware installed in the Reader? Sony made a new release, and it's possible that the original earlier version doesn't support FAT32 and/or the 4Gb cards. If you have installed the Connect Reader software on the your PC, when you connect the Reader and go online it should give you the option of updating the firmware to the latest release.