Well, I can't really understand why this happen and the offered solutions sounds more like use a sledgehammer to crack a nut.
(sorry, it is maybe a bit unfair comparison, especially as I don't know anything about the book)
We had in the past as well huge omnibus ebooks that we convert in both directions between EPUB and Mobi using a 32 bit OS with calibre and mainly without memory problems. Hm, we are talking here about a 375 page EPUB as source.
I can't really believe that calibre isn't able to convert a "standard" sized book from EPUB to Mobi with calibre 32 bit and the only possible solution is to take a 64 bit system.
For me there is something else wrong. Maybe the book or something in the 32 bit conversion program itself? Don't know. The OP can check as first step the EPUB for errors with the calibre editor. If the book seems to be ok and the conversion error is still there, he should make a bug report and attach the file so that Kovid can take a quick look on it.