Hi Jeremy,
Yes it is fine for the baby to eat grain. Is he also getting milk? As far as the older two are concerned Garden clippings are great and I think a little grain for the next year (until they are two) and I mean like a handful each twice a day would be fine. However you want to balance that with something that has calcium in it. You've probably heard about urinary calculi... that is what you are trying to prevent by not feeding too much grain. Wethers don't need that much energy and grains are also high in phosphorus. For calcium to balance things I'd feed beet pulp. It is high in calcium but not high protein (which alfalfa, another calcium source, is)
You didn't mention any minerals so I'm not sure if they are getting those. You need a loose mineral that isn't a "sheep and goat" mineral (too low in copper)... something like manna pro or onyx.....
Grains = sweet feed, corn, oats, barley etc.... In my opinion not necessary over a year old (although mine got some till quite recently because... well they liked them so much.....)
When you say "today is was sweet and corn" it seems to imply that yesterday they got some other grain combination. I would be careful with that because an inconsistent diet/sudden change in diet can cause serious havoc with the rumen. What sweet feed are you feeding? If it is goat specific then I think I'd just feed that (at the rate that is recomended on the bag). If it doesn't say that it is for goats you could post the nutritional analysis and I'm sure someone would be willing to take a look for you to see if it an appropriate feed.
You could also just feed plain grains but I'd be a little concerned that the diet would be unbalanced and frankly for me it is easiest just to buy a premixed thing.
Anyway I hope some of that helps.
Here is some more I've written about feeding at various points and, down at the bottom two links that might be helpful...
https://sites.google.com/site/2creeksgo ... th/feeding
Goodluck!!
M.