Thanks for the hint! I tried some books in epub and mobi format and coolreader did not show a TOC for any of them.
Now converting them to fb2 indead gives me a TOC. The TOC does not contain the chapter names but only the page numbers but I can live with that
