Topaz is basically a set of images (SVG) with OCRed text n the backround for searching, or so I understand. We hate Topaz, but some books are only available through Amazon in Topaz because they are old and were scanned rather than created with HTML. You should be okay deleting them, and yes you will have to delete them manually as calibre wouldn't know which images are used and which are not.

Calibre also has a setting to make sure HTML files do not exceed the size limit. If they are too big it can break them into smaller files.

Your book was probably just poorly formatted and at some point the reader puked on it. After all, Topaz to ePub is quite a stretch and a lot can get screwed up in the process.