Try the sigcap program (script) for a baseline comparison of speed and responsiveness (lack of lag), and to see one way to prevent the framework from stealing your touchscreen events:
Especially notice that I have a "granularity size", which could also be called "sensitivity", or "hysteresis threshhold" depending on your background and experience.
Because the touchscreen puts out 0-4095 values for each axis, this can generate WAY more events than are useful. I filter the events down to only process events that go outside the bounding circle that is "granularity size" radius from the last captured finger position. You can adjust that value. Smaller gives a lot more capture points for higher resolution if you need it. I would not go below 4, personally (no need). I use 6 for my onscreen keyboard code.