It might be helpful if we all tested the same book.
I downloaded a public domain non-DRM book from Project Gutenberg. Loaded it directly into Marvin; did not run it through calibre.
I selected "The Adventures of Tom Sawyer by Mark Twain", EPUB with no images (301 KB).
https://www.gutenberg.org/ebooks/74
And ... I could NOT change text size unless I turned on Publisher's Layout.
So, the book does seem to matter. The first book I tested was "Zoo2" by James Patterson and it worked fine.