Keep in mind that twobob did this VR project several years ago - -
It probably wouldn't hurt to use updated sources.
Some words on how it works:
http://cmusphinx.sourceforge.net/wiki/
Of interest, Vague is still acknowledged (as a mobile application):
http://cmusphinx.sourceforge.net/wiki/sphinxinaction
(Hmm, I guess I should do something about making that link work again.)