For ARM you need this file arm-kindle-linux-x-tools-glibc2.5-gcc4.2.4.tar.gz (45741995 bytes). Just unpack it's "arm-kindle-linux-gnueabi" subdirectory into /usr/local/arm and add /usr/local/arm/arm-kindle-linux-gnueabi/bin/ to PATH and everything will be OK. (compile with make HOST=arm-kindle-linux-gnueabi)
For emulator, on my system (FC12, 32bit) everything compiles fine, with no adjustments to the Makefile.
Just to add that the x-tools file is very hard to find, but you will find it in one of those "forbidden to even mention" places
I have no idea why, but this fact prevented me from hosting it on my own site, just in case... The x-tools toolchain is preferable because it already contains the correct sysroot stuff for the Kindle, so you don't have to pick the libraries from Kindle's actual root manually.
And, btw, if you are interested in just a PDF/DjVu viewer (without builtin CoolReader) then you can build the kpdfdjview branch (git checkout kpdfdjview) which has fewer dependencies and builds much faster, resulting in a smaller program.
Also, to build an installable package you need to do "make HOST=arm-kindle-linux-gnueabi customupdate".