To keep objects together, enclose them within a DIV or P tag and use
inline-block.
That is,
In your CSS, have something like this:
.image-and-caption {
display:inline-block
}
Then in your html:
<div class="image-and-caption">
<img alt="text" src="image1.jpg" style="height:90%; text-align:center;"/>
<p>Caption text</p>
</div>
Leaving the "height" at 90% (or whatever is appropriate) will leave enough room for your caption text.