Yes heavy wormload or parasite load can cause infertility. Sometimes that infertility is temporary and sometimes permanent. Certain mineral deficiencies can cause this as well. Any severe lack of condition can cause it.
Are you sure that your bucks coccidia load has been fixed? Coccidia a protazoa will not be killed by wormers it would need a sulfur drug to treat it.
I have a doe that had a extreme worm/cocci load when I got her, I tried for 2 years to get kids out of her after I got the problem fixed, she finally had twins, but she is a full size alpine that only got 26" tall, tiny thing, but the favorite. She is healthy now, but it can happen, not only infertility, but also stunt them.