I have heard several ways of avoiding gaoty tasting milk. I had a Mennonite woman tell me she puts ice into her milk bucket. I have heard people say to separate the buck from the doe.
I only drank goat milk once and is was delicious. I ate goat curds once and they tasted goaty.