There seems to be a huge difference in size between XS and S too! At least on some books. Don't forget that if you go to landscape, the font size increases autocrappily. That makes it even more fun to find a perfect font size(s).

I converted a book several times selecting a different size for the Base font option with each conversion, then compared the many conversions and chose the one that most pleased me. But there is still too much difference between sizes IMO.