I think you're going past the maximum of 64K PDB Records in a MOBI file:
https://wiki.mobileread.com/wiki/MOBI#PalmDOC_Header.
Looking at that, an uncompressed MOBI file can't be larger than 4K x 64K = 256M. I'm not sure what Calibre does; if it uses Huffman compression, it should fit, but perhaps it doesn't.
Edit: Ninja-ed :-)