It could perhaps be the hormone residues present in mainsteam dairy products - given that the dairy industry in Aust adds large amounts of synthetic growth hormone to cow feed, which subsequently passes into their milk. These hormones (which mimic the female hormone estrogen in the body) are banned in various countries overseas - the UK, and various countries in Western Europe, but still used freely here. The dairy cows develop infections (thrush, mastitis etc) from overuse of the hormones and are then routinely administered antibiotics - the residues of which have also been tested as present in mainstream dairy products.
These hormones are linked with nausea, precocious puberty (early breast development & early onset of menstruation in girls), low sperm count, breast cancer, just to name a few!! The antibiotics are cheifly blamed for interrupting the balance of good bacteria in the digestive tract (probiotics), and antibiotic resistance.
Maybe you will be able to tolerate organic dairy produce - as the use of synthetic hormones and antibiotics are prohibited in organic farming. Just a thought.
If not , don't be concerned - there are plenty of ways to ensure you get enough calcium. You will be amazed how much calcium is in the humble sesame seed!! Get yourself some Tahini - the sesame seed spead. A much healthier alternative to butter or margarine!!

XX