Replace skiing with a myraid of other kids sports and you end up asking the same question.
But it seems to me, especially living out here in Vancouver where the local feeder hills are all upwards of $60 a day, and Whistler Blackcomb now wants north of $100 a day, $175 for a single day lesson with lift for your wee-little one, Skiing is rapidly becoming the sport of the very wealthy. And since the literal price of admission is so high, that disqualifies the ones who might want their kids to do it just for fun.
The attitude I've seen from a lot of racer parents is that if you're paying big money to have your kid be a ski racer, you want your kid to succeed. And you want your kids to have the best coaches. And the kids who want to just have fun find that in this high pressure environment, it just isn't anymore. Especially if you're tired of loosing constantly. At the hill I work at, I saw a coach from the local ski team ripping into a group of 10 year-olds for having the audacity to have gone over a "jump" on the side of the run. This club is moderately priced at only "$5000" a season in fees, (not including passes, equipment, and so on) but it's still not cheap by any means.
So that leaves just the hyper-competitive parents and their kids, the ones who see no problem with paying tens of thousands for it. But does that mean the best athletes get produced? Probably not. There's probably a lot of phenomenally talented skiers out there who would never make it on to one of these teams due to financial constraints, or a desire not to spend one's entire life doing nothing but skiing. I mean, it's the best sport in the world, but still...