Hello @beeace336
You don't need any plugins to change file extensions.
I've had a look around the Conversion settings screen, but could not find a mention of plugins, or any setting for html<>xhtml settings. Not sure where you saw the requirement for a plugin.
But the fix is simple...
- Open the epub in the Calibre Editor,
- highlight the files that need changing,
- then from the context menu, select "Change the file extensions for the selected files".
- Then type xhtml in the box when it pops up.
- Save the file if you are happy with the change.
See the image...