Check the Debian source package for that compiler -
You will probably find it includes patches, different than those you may have applied with crosstool-ng.
*AND*
You never mentioned what compiler version (and source) you built with crosstool-ng - -
So we may be comparing Apples to Oranges.
- - - -
PS: crosstool-ng is deprecated, use current release of Buildroot (and the 'make toolchain') command if you want to continue building your own cross-toolchain.
- - - -
Hard Float vs. Soft Float **should not** make a difference for building the Linux kernel (it does not reference external libraries).
But it will make a big difference when you start building userland firmware.