In principle you are right. That is the way how you should include CoolReader. Although for this special case I have my doubts if it is a wise idea not to use FBReader as the default for fb2-files. At least, FBReader has been invented for this purpose. But if you think CoolReader works better for you for fb2, okay.
An old discussion of the extension.cfg you will find here:
And maybe the CoolReader site
might be interesting to learn which formats CoolReader supports. I think it works well with chm-files too.