I kinda think that a mute on a device that has no external speaker is largely pointless, I'd rather see this "hold down" trick pause/un-pause, I think.
The only way I've found to completely turn the music off was to completely remove the sound files.
(I don't want another music player, anyway
)