Is it possible to have an allergic skin reaction to rosemary bush/plants?

If you are allergic to touching it should you refrain from eating it? Even if you've eaten it before?