Kovid's solution should work, or what about trying:
Code:
# Chapter 4
Blablabla...
# Chapter 5
<a name="anchorForChapter5"></a>
Blablabla...
So that the anchor is actually in chapter 5? This sort of anchor can be used more generally than just TOC entries.
If this skips the chapter header you could try:
Code:
# Chapter 4
Blablabla...
# <a name="anchorForChapter5"></a>Chapter 5
Blablabla...
I use calibre to convert markdown to ePub for a Sony reader and find that it does jump to the correct position of anchors within a page.