I've actively sought used books off ABE a few times, both before and Amazon bought them out. Prices will vary according to scarcity and demand, with scarcity being the main driver.
As WillAdams said: what the market will bear.
Two examples:
When I first started targeting SF books in my early teens I compared notes with my school buddy who mentioned his uncle hadleft behind a few paperbacks. I loaned him mine, he loaned me his. I had Heinlein and Blish; he had a couple of ACE doubles from the 50's, old even then. Fun reads. And being among my very first genre reads they stuck with me. Those ACE books were dirt cheap and relatively high volume for the genre but when I went looking for copies for myself, 50 years after release, there were maybe 5 copies on the market in the whole planet. And not in very good shape. They still commanded 20x the original list. (Which was fortunately in the $5 range.
) For my purposes, the fact it pretty much unbound the moment you opened it was expected and no problem.
A second case was a hunt for Chad Oliver's SHADOWS IN THE SUN. Also a book from the 50's. That one I read from the university library. Like most 50's SF novels it saw a hardcover print run of maybe a thousand copies total. There was an early 70's limited edition reprint because Oliver is one of the seminal soft SF writers. When I went hunting, the most readily-available edition ran $60 and there were maybe 5 or 6 copies available. One vintage edition ran $600. I found a decent one from another, more recent, reprint for $22. And I was lucky; it was the only copy that low.
Used book prices can range from bargains to outrageous depending on supple, demand, condition, and significance to the likely buyer. When I ordered the Oliver book, I picked up replacement copies of a couple of my old Heinlein psperbacks for $1.50 each. Excellent condition, too. Same edition I had from decades past, a lot cheaper than the current in-print edition because Heinlein is always in print. And now he's in nicely-priced ebooks. Unless it's a vintage hardcover edition there's not much point to go used.
But for lesser-known authors and rare editions the price can go very high simply because there are few copies to go around and the market will sooner or later bear a lot.
When you get down to it, they aren't priced as books but as collectibles.