It depends on your target. If it is ePUB3, you should use MathML. If it is ePUB2 (most supported), equations are best of as images (png or svg). I personally use SVG, since I can make that scale-able to the font size. I use the
codecogs site for creating SVG equations.