I guess I'm a spirit person, too. DRM is like a speed bump. There's nothing sacred about either; they are just devices to serve a purpose. DRM is a weak device, at that.
People whose ethics are even more effective in achieving the purpose of such a device shouldn't feel guilty for removing it, especially if not removing it would prevent them from fair use, an injustice all its own.
I wouldn't buy a DRMd ebook if I thought I'd feel guilty for stripping it if I ever switched readers.
Of course, I've never been one to put much faith in laws determining right vs wrong. Legal technicalities get real criminals off, while laws that haven't matured with the times can make a criminal out of the good.
Did you know that owning an Encyclopedia Brittanica or taking more than 2 sips of beer while standing is illegal in Texas?