This is a general problem of any zoom mode. We definitely need a way to get a button to navigate effectively inside the zoom mode. I think the joystick could serve this purpose by not only moving the view by some pixels, but jumping to the next part of the page that is not in the current view.
At the moment, the only way to achieve a similar behaviour is by using the pen by clicking and dragging. It's pretty obvious, that pen navigation is not very comfortable in many situations. So this can't be the only option in the long run...