Also, if you ever want to drive your goat, you will need a halter. It also helps for controlling your goat when he gets big. Even a well-behaved goat may need to be contained sometimes, like when you're training him to stand for hoof trims, bathing, clipping, or teaching him to do obstacles for a pack class. In these cases it's much easier to control the animal with a halter, plus a halter won't choke him if he struggles against the restraint.
I also keep Cuzco on a leash when I'm walking him in areas where there is going to be car traffic (such as when we go to the P.O. and the grocery store). He's so huge that if he's only wearing a collar he can easily pull me over if he decides to go his own direction (in fact, since he's been harness trained, he tends to lean into a collar if I put pressure on it). The halter, on the other hand, can be used to turn his head, which means he can't brace and pull me around by it like he can with the collar.