I was told it's generally not an issue for adults as it is so prolific most of us will have been exposed (and infected) in childhood.
I wouldnt' worry about washing your breasts - breastmilk has natural anti-infective properties so is kinda self-cleaning.
Hope your boy is all well soon