There are two ways of coding "Table of Contents" data.
One way is to code it in HTML as part of the front matter in an ebook - exactly as it would be in a printed book - with each entry in this TOC coded as a link to an HTML anchor embedded at the appropriate point in the book content. This is often referred to as an "inline TOC", and is the primary method of providing a TOC in MOBI/Kindle ebooks.
The other way is to code it in XML in an NCX file (Navigation Control XML) which is then compiled into the resulting ebook. This is sometimes referred to as a "logical TOC" or "NCX TOC", and is the primary method of providing a TOC in EPUB ebooks.
Unfortunately, those who mostly work with MOBI files (or who are not yet used to working with ebooks, and are thinking in terms of MS Word, Open Office, and other print-related formats) tend to refer to the inline TOC as "the TOC", while those who mostly work with EPUB files tend to refer to the NCX TOC as "the TOC", leading to much confusion.
In an EPUB reader (device or application), when the user chooses to view the table of contents, the EPUB reader accesses the information in the NCX file, and generates a table of contents from that information, and presents it to the user.
In a MOBI/Kindle reader (device or application), when the user chooses to view the table of contents, the MOBI/Kindle reader looks for a "guide" data item pointing to the location of the inline TOC in the ebook HTML content, and jumps to that location, displaying the table of contents to the user. That "guide" item is originally coded into an OPF file - an XML file containing metadata about the ebook - which is then compiled into the MOBI file. The data in the NCX file, on the other hand, is used by the Kindle device and applications to enable "skip to previous/next navigation point" functionality (e.g. the ability to use the 5-way control button on the Kindle to jump to the previous or next heading or chapter). The navigation points coded into the NCX file are also displayed as small dots or tick marks on the progress bar displayed at the bottom of the reading screen.
The reason you find that the TOC created by Calibre when converting your HTML files to MOBI format can be "seen" by the Kindle, is because Calibre adds the required "guide" item to the MOBI file, pointing to the TOC it created (which is an inline TOC, just one that is placed at the end of the book content). And the likely reason the Kindle isn't seeing your hand-coded inline TOC is because there is no "guide" item pointing to it (possibly no OPF file at all?).
You can certainly hand code an OPF and NCX file to accompany the HTML file(s) for your ebook, and you might decide to do that a couple of times to become familiar with the various files that go into an EPUB or MOBI ebook, but in the longer term, I'd definitely second the recommendations others have made regarding using Sigil and Calibre. Although your final goal is a MOBI version of your book(s), it's worth using EPUB as an intermediate stage, since it is easier to ensure that all the component files are present and correct, and to edit the content of these files if required.
I'd suggest switching off the "HTML Tidy" feature in Sigil to minimise the changes it makes to the HTML code (given your desire to maintain as much control as possible over the actual code
). Add your HTML file(s), then use the TOC editor to create the logical/NCX TOC, and the Metadata editor to add the required metadata (title, author and language are the minimum requirements). Sigil will take care of creating the NCX and OPF files - you won't see them in the file list on the left of the screen, but they're there in the "package" that Sigil is creating which it will zip up into the final EPUB file.
(Note: As you add your HTML file(s), Sigil will parse them, and automatically find and add any external CSS files they link to, and any image files they refer to. Unless you have included your cover image as the first page of the book content, you should then add the cover image file in the "Images" sub-directory in the left panel, and use the "Add Semantics" feature (as described below) to designate that image file as the cover image. Sigil makes its own copies of all of these files - your original files will remain where they are, and unchanged.)
There are two routes to getting the required table of contents "guide" item into the OPF file:
- In Sigil, split your HTML content into several files, one for each logical section of the book (e.g. title page, copyright page, table of contents, dedication, foreword, each individual chapter, etc). In the file list on the left of the Sigil window, right click on the table of contents HTML file, select "Add Semantics", then designate it as the "Table of Contents". Save your EPUB file, and import it into Calibre.
- If you prefer to keep your HTML content in a single file, add an HTML bookmark at the start of the table of contents in the HTML content. When you import the EPUB into Calibre, use Calibre's "tweak epub" feature to open up the EPUB, then open and edit the OPF file and look for a section in the file bounded by opening and closing <guide> tags, and add the following code between those tags:
Code:
<reference type="toc" title="Table of Contents" href="html-filename.html#toc-anchor"/>
(If there isn't already a "guide" section in the OPF file, simply add <guide> at the start and </guide> at the end of the code shown above, and add it after the closing </spine> tag.)
Save the amended OPF file, then re-form the EPUB.
You can now convert the EPUB file to MOBI format in Calibre, and the existence of the "toc" guide item should ensure that Calibre doesn't automatically create and add another inline TOC to the ebook. When you open the resulting MOBI file in the Kindle, the "Go To... Table of Contents" menu entry should work properly, and cause the display to jump to your hand coded inline TOC.