No need to remove anything else.
The battery is held in place with some stick stuff, so if you want to remove it you have to pry a little bit. But, you don't need to take the battery out to disconnect it.
Here's a photo of the unit once the cover is off. Battery is in the top right.
And the battery itself.
