I don't feel guilty about not reading non-fiction more. Sometimes a fiction book leads me to look for a non-fiction or i pick up a non-fiction when I see something that interests me. But guilty? Definately not. What is the point of me getting more non-fiction (either bought or from library) and I don't enjoy them and therefore won't read them.
For me, reading is mainly about escape and relaxing.