Played more with it. Sizes 4,5,6, and 7 display body the same on KT (5.3.2) and PW (5.3.3). KT is a tiny bit smaller (almost not noticeable) on screen than PW. So the lower limit appears to be a certain size not in pixel, but maybe points? Have to keep digging deeper into fontconfig - maybe there is something. If the limit would be pixel, than the PW should display smaller.
This tested on both mobi and azw3, same result.