Yes, it should be perfectly safe to format the drive. You will of course lose all documents and possibly some hacks (jailbreak, usbnetwork).
So as an added safety measure, I suggest you copy all files before reformatting.
I just tried it on my K3:
Code:
sudo umount /dev/sdb1
sudo mkfs.vfat -F 32 -n "Kindle" /dev/sdb1
(for you Windows people, that's the equivalent of ejecting, then reformatting, the partition).
Then I rebooted the Kindle. It *did* come back with (only) the Oxford dictionaries listed, so they're obviously stored somewhere else.
Finally, I plugged it in again and copied back all the files, and now it looks exactly as before.
Bottom line: yes, it's safe to reformat.