I don't believe there's any such beast as a "non HC" 4GB card. Above 2GB all SD cards are SDHC, to the best of my knowledge.
Yes, the Reader will show your card as 4GB, but I'm certain you'll find (as many others have) that you won't be able to access more than 2GB of data on it. Any files that you add to the card above 2GB simply won't show up. Try it and see.