Cow milk protein intolerant household here.

With me, it's runny noses and hayfever signs when I had cowmilk as a child - and if I overdo it now.

With DS, it's agonising stomach pains and eczema flare-ups with dairy. He would just be screaming in pain all night. He is starting to tolerate a little dairy, but I try to keep him away from cow milk as much as possible.

We use goat milk, which doesn't give us this problem. But does give us another in that DH really hates goat milk. But switching to goat for a trial won't hurt.