Sounds like you have it pretty much all figured out!

Other people had the problems you mentioned -- loss of library, SD card issues. Keyword searches of this forum (library, fat32) might dredge up some answers, if you haven't already found them.
Enjoy!