[Quoth]

Letters, thoughts, Telegrams/Telex, typewriter, poems/lyrics, sub headings, marginalia, inline notes all might need fonts not built in, or at least differentiated. Even the horrible mobi format does:
Serif, Sans-Serif, Monospace and each in: normal, italic, bold and italic-bold.
An epub CSS defining a font-family always should have the fall back too:
generic-family - The name of a generic-family, which is "serif", "sans-serif", "cursive", "fantasy", "monospace".

Any CSS for "cursive", "fantasy" should be edited to have some mix of Italic on Serif or Sans as appropriate so that the paragraph or text span still "works".
Anything using a monospace embedded font needs care for Kobo epubs and conversion to mobi to ensure a monospace font is used as things like ascii art or "white spaced" layout only work in Monospace.

So before removing embedded fonts make sure there is the correct fall back.
Be wary of removing any embedded symbol or non-Latin/Roman font.

Edit: Some old Sony ereader simulate italic with oblique, which is just a slanted version. Real italic doesn't use all the same design of characters as the normal version. Similarly some ancient systems simulate bold by repeating normal with an offset!