Coffee tastes so much better without milk, so try that. (Or express your own for one coffee a week.)

Liebling has goat milk and loves it, thinks it tastes the best. When I'm pregnant and Stormy's feeding then I'll be on goat too (I had cow intolerances as a child too) and I have no problem with it. I find I don't notice the "goaty" taste at all in things like white sauces and cheese sauces when I cook with it, but too much uncooked stuff then I do notice a slight aftertaste: nothing while I'm drinking, just a brief kick when I've finished.

Start with mild goat cheese though, or sheep cheese. There can be a very strong aftertaste with the stronger stuff that some people don't like.