I've found that even if I use 12-point for the body of the book, if the book contains any fonts bigger than about 24-points (like for the titles or chapter headers), then after I transfer to the Kindle, it uses the smallest font size when you choose either the smallest or second smallest options on the Kindle. It does start to adjust up when using the 3rd, 4th, 5th, and 6th sizes.
In other words, I used 48point fonts for the chapter headers, and 72-point font for the title. I used 12-point for the body of the book. Then, when I transferred it to the Kindle, it displayed in the 7-pt font, even when I chose the 9-point font size. Once I dropped the fonts back to 24 points for the Title and chapter headers, the problem went away.