I weigh up the needs when cooking with high fat/low fat, for the kids for the adults and what it is i'm cooking. It may also depend on what i have more of at the time, full cream/no fat milk. I read or heard once that what they do to things to make them 'low fat' (for example cheese) doesn't necessarily make low fat the better option. So i do not buy low fat cheese, we have both milks as i don't like the thickness of full cream but i will cook with it.

I know you said you are staying away from dairy at the moment but in general i tend to go with common sense and convenience. It may not be considered nutritionaly a good way of looking at it but i know majority of my childen's diet is excellent so i can give and take here and there.

HTH