The ToC view is not "live". You have toeither close and re-open it, or right click and choose refresh.
EDIT: Actually, currently deleting a file does not automatically remove refernces from the toc.ncx, so the toc view will not change even if you do refresh it.
EDIT2:
https://github.com/kovidgoyal/calibr...7e7f24f22638e1