The most important factor to me is access to multiple formats. I prefer reading PDF (formatted as the paper book) on a tablet, but read the same books in EPUB on other devices. Ebooks.com allows download in multiple formats on the same purchase and I often pay a premium (ebooks.com rarely offers discounts) for the privilege. Closely related is that the bookstore offer PDF editions; few do and I'll pay more for a PDF than for an EPUB.
The next criterion is that the seller not alter the publisher's formatting. For that reason, I won't buy from Google Play at all nor from Kobo if the book is only available as "kepub".
Third is lack of DRM. I still buy DRM books, but if it's available without, I'll buy that way, even at a premium. Christianbook offers many books without DRM that others put DRM on and in that case, I'll preferentially buy from them.
Finally, I don't want a "one-click" interface. Despite owning and actively using a Kindle, I stopped buying from Amazon entirely when they switched to the one-click interface for ebooks.
If a book is available in EPUB only and is non-discountable (Agency), then I buy it from whomever I've "bought" the most free books from lately. For anything that doesn't fit any above categories, I buy from whichever store is cheapest (usually Kobo with discount codes).
One more thing that I want to mention is that, while most people seem to think ebooks must be cheaper than pbooks, I disagree. I gladly will pay (and have paid) a premium for an ebook over a pbook.