I'm the same as Kazzo, relaxed about what I eat and get people berating me. You truly cannot please everyone, and frankly you don't need to.

Eating/not eating soft cheese - there are worse things we could do! Smoking & drinking come to mind. You don't have to explain yourself at all to these people.

Sorry I don't know what you could say back, cause I'm relaxed with the food thing and haven't dealt with anyone who anointed themselves to be my doctor.