Typically, these are not included in the ePub, and then they are imposed by the reading software. Some of them do them differently than others, hence what you're seeing.
It is also possible to put a "page-map" inside the ePub to force the numbers to appear at certain points, though strictly speaking this is supposed to be an Adobe-only extension, and it'll make your ePub fail an ePub check. It works though, although some reading software (calibre, Stanza) will ignore them.
You can read about it in this thread:
https://www.mobileread.com/forums/showthread.php?t=53195
LoganK is incorrect technically; you can get rid of these, but there are significant costs. See the thread.