OK, try this:
1. Install
Mobipocket Reader, if you already don't have the latest version.
2. Launch that software reader, then drag 'n drop your .chm file onto the Mobipocket Reader screen.
3. It will convert it to .prc; check the bottom of the reader's screen for staus updates.
4. Go back to library/ebook view, find it. Click it once, then right menu button and select 'Open containing folder'
5. Feed that file to Calibre and have it convert the .prc to .lrf/.epub.
Note some .chm's will work better than others.