Yes, sorry -- I was unclear. I was talking about the MP3 player application on the Sony Reader. Is that not what you're talking about?
The application doesn't have a "stop", it only has "play" and "pause" -- if you watch it when you "stop" the playback, the display doesn't say that it's stopped, it says that it's
paused.
In any case, when I do as you're describing, and stop/pause MP3 playback, it holds the place just fine. I haven't played with it a great deal, as I don't use the thing to play MP3's, though I do anticipate that I might use it as you're describing for audio books.