As I see it, this issue has no solution.

Like you said, converting PDF files to tolerably formatted EPUB files takes too much time. PDF files are
fake e-books, but many books are only available (reliably, or at all) as PDF files, unfortunately.
Moon+ Reader Pro is my Android e-reader of choice, but I've never even attempted to use it as my PDF reader. You might give Mantano a try, but I wouldn't be too hopeful about that, either. PDF support in what are primarily
EPUB readers often leaves a lot to be desired.
I use ezPDF Reader as my PDF reader on Android; it seems to be the best of the lot, and it's definitely a decent app. However, I find it lags in terms of overall quality and functionality behind GoodReader, the best PDF reader on iOS.