@Kai771: Just for confirmation: run file on a binary built by the Ubuntu/Linaro TC: What minimum version of the Linux kernel does it report? [ie. ELF 32-bit LSB executable, ARM, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.22, not stripped]
I'm guessing something way newer than the 2.6.22/2.6.26 of the K2/K3, which would explain why it's segfaulting on a stupid stuff: ABI mismatch. I remember that kind of error being somewhat more user-friendly on x86, where the loader actually tells you why it can't run a binary instead of segfaulting in your face, but, hey...
As for O2 vs. O3, it has the potential of starting a holy war, but basically: I'm not a huge fan of using O3 as the goto 'default' setting. Use it when you know it's gonna be useful, otherwise, it more often than not *lowers* the performance (especially on such a tiny CPU), and almost always ends up generating larger binaries. That said, Linaro does recommend defaulting to O3/Ofast on ARMv7 with their TC.