Yes, see Pablo's Quick-'n-dirty EPUB tutorial in our wiki:
EPub_Tutorial#Footnotes
It was written for Sigil before calibre included an editor, but is still very useful as a general EPUB editing guide, and includes a section describing how footnotes should look.
Also, w3schools is an excellent reference for HTML in general, and what you are concerned with here is the HTML "link" tag, explained here:
http://www.w3schools.com/html/html_links.asp