Uncomfortable, yes (I much prefer paper back or my Kindle to a hardcover). Funny or disloyal, no! Reading is for your enjoyment, so if you are in the middle of a hardcover, so be it.
Knowing me, if someone gave me an odd look that really bothered me, I'd probably say something like "Its a book! Try one!"
