Publishers don't seem to have too much trouble putting one or the other on the back cover. It's just stores and libraries that pile them all together.
I think I read both fantasy and SF for the ideas. I want flights of imagination and a good story. I can do without one of the two, but not both.
I think claiming SF is possible is a bad way of looking at it. Time travel? Psionics? Or are they relegated to fantasy when they're deemed impossible? SF is about the present. I think the key difference is the kinds of stories you can tell, and the kinds of people that inhabit them.
I don't especially have a problem with contemporary fantasy. I imagine I'd like the best of it. My issue is that I like epic fantasy and I can't find out what's hot any more, because the charts are flooded with the urban supernatural variety. Surely that's what the genre distinctions are there for? So fans can find what they're looking for?