It seems that the sony reader is suffering from the same fat32-limitations as the PSP.
After using this trick:
http://hightech.afmag.net/make-4gb-m...n-default.html the speed of my 4GB card is on par with the 1GB ones.
Well, at least sony is consistent with their errors....