With breastfeeding, do you need to avoid foods the little one could be allergic to?

Eg; DH's family has a heap of people allergic to shellfish, we don't know if Munchkin or DH are allergic though.
Do I need to avoid shellfish whilst breastfeeding little Nao because of this?
It's probably a silly question but I honestly have no idea how it works.

Thanks in advance~