I guess it's definitely possible to be allergic to rosemary. It would make sense to avoid eating it if you believe you have an allergy