First of all, I'm going to dash your dreams right away: SDL2 is going to be a no-go, it no longer has a linuxfb video backend.
Secondly, cross-compiling Python/C extensions is... not a fun experience, cf., the rants about that in the Python section of the
x-compile script.
numpy, on the other hand, is common enough that it would make sense to bundle it.