Thanks very much.
I managed to change the metadata using Calibre.
Here's the solution for others with the same problem:
In this case, using Calibre, before converting, just edit the metadata and set the language to Japanese. This step wasn't available from the conversion settings dialogue box, so it had to be done prior to conversion. After conversion, the Kindle will be able to recognise the text as Japanese text.