whilst i don't think your 3-star reviewer was fair in grading your book based on whether it has a TOC or not, i have to admit i like to see the TOC there even if i don't use it much.
i think it helps me get a feel for the book once i open it. first is the cover, then the TOC (is it a 20 chapter book? a 70? etc) maybe even if it's just a preview...
and then keep in mind that reviews are subjective