You can't put a battery designed for another device into the Voyage. Battery space is very critical and the slightest bit of extra thickness will cause the side buttons to not work. The controller board with the battery needs to be designed to work specifically with the Voyage.
Check out this post and see if SKK still has any extras. The ones he sold me worked for me.
Broken Kindle Voyage Screen with Good Battery
The upper back portion is fastened down with clips and double sided sticky tape. You may need to replace that tape carefully when reassembling. I just used carefully cut pieces of double sided tape. The back of the case is not glued down. Once you remove the small upper back portion and the 4 screws inside the upper back just slides (with great difficulty) down and off. However the battery is glued down onto the Kindle. You can probably just use double sided tape to hold the replacement battery in place.