One music program you should take a look at is Mediamonkey. It is a great program for people who like organizing their mp3s to the point of OCD! It's Windows only so if you're on a different platform I can't suggest anything else.
I'm a huge fan of both Mediamonkey and calibre and think there's some definite overlap in what I like about each. They both can handle huge collections in a way that's organized and help you find the file you're looking for.
MM has a VERY active community in their forums which helps if you have any questions or looking for how-to information. It's also scriptable and has a huge number of plugins and scripts available.
No - I don't work for them and I'm not trying to spam for them - I'm just a long time user and fan of both MM and calibre and think that MM does for music files what calibre does for ebooks!
Give it a try -
www.mediamonkey.com