[NiLuJe]

You're looking at it the wrong way around.

There's *always* a translation necessary, the touch layer *never* matches the fb.
c.f., Plato's input code for an idea of how "fun" this is to handle, or my first answer on github when I hinted that this was liable to drive someone mad for some other examples).