The thing with fiber animals is that you need to have a big operation just to break even. I can sell washed mohair for 25 bucks a pound easy and if I get it processed, way more. BUT, your animal produces maybe 16 pounds a year and if you sell it all that is only $400.00. When all year round you are paying for feed, medication, processing fees, washing supplies, shearing fees, vet bills, stud fees, show entry fees...the list is endless.
Plus, in order to sell that 16 pounds you have to do some traveling to fiber festivals unless you can find lots of people in the area that will buy. That's gas, food, renting a space to sell on...
Not to mention, all the time that go's into shearing, washing fleeces, skirting fleeces, clipping hooves...
It's certainly not for everybody and it is HARD to break even, believe me...
But don't get me wrong...raising animals for fiber is fun and rewarding, I'm just saying they are not ideal for making money with on a small scale farm. If you got two high quality does and sold offspring every year, that could help with things but you still got all those dang expenses.
I think Hog raising seems like a good use for one acre. You said you are not into selling for meat, and I feel the same way, that's why I have angoras and not boers. But, personally I have never felt attached enough to a hog where butchering would not be an option. They just aren't like goats in that they have unique personalities and you get really attached to them. That's just my opinion, I'm sure some people feel differently. I suppose you could purchase a couple Sows and sell feeder pigs?
Good luck.