You sir, are a genius.

It brought the patch from 22MB to almost 500KB.
Making the patch out of the unzipped copy did the trick. I also assumed it had something to do with the loop mounting. Once I test this version of the image, I'll update the patch. I've added USB Keyboard support on boot.