If you are really, really lucky, you can just delete the image files in the file browser and the alt text will take over. But as others have said, it may not work in some e-reader, or it may be ugly, or just not work at all.
I usually find some combination of regex S&R and manual editing to do the job. Like maybe search for <img.*?> and replace with <h1 class="chapter">Chapter </h1>, and then go through and number the chapters manually (using replace all only if chapter heads are the only images).
If you actually like the alt text available, you might get fancy enough to pull the text out of the "alt=" part of the <img... line with a regex expression and then stuff it into a replacement string. Depends on the coding you have how hard that would be.