The batteries in the Pocket Edge are not replacable*, just like in about 99% of the devices nowadays. As for cases, also a bit hard to find. I took mine to about 4 different store and eventually settled on a Targus case ment for a 10" netbook. My newly aquired "Man Purse" (or murse

) is a bit big just for the PE, but holds the charger, cables, my ipod etc. as well.
"Must Have" apps are a personal choice, I find. Mine are:
Note everything - uber note app
Tag ToDo - todo list (with tags! natch)
Thinking Space - mind mapping
Shortyz - crossword puzzles
cool reader - ebooks on the LCD side (for nighttime

)
stopwatch - self explanatory
250+ Solitaire collection - also self explanatory
Form Builder - simple database, no encryption but export with CSV.
*my Palm TE2 had a non-replacable battery as well, but when it died I figured out how to crack the case and change the battery. I found a battery the same size and specs in the internet. The TE2 survived the operation and lived another year and a half until it meet an unfortunate end by falling in a toilet.....