We started my oldest at 4. Now at 9 he's always in the park doing tricks, and I will admit he's pretty darn good!
Now my daughter is 6 and we will try again this year. We went up last year for a lesson. We never even made it to lesson. As soon as she was all ready to go, I take her outside to put the ski's on, and you'd a thought I was beating her with the ski's she screamed so loudly. So that was the end of her ski season. She never wanted to go back, we'll see about this year. I find it pretty funny that I try and get 75+ days a year and she really has no desire to ski what-so-ever. And I'm not going to force it either. If she wants to great, if not well I guess we'll find something she'll like. (We are going to try snowboarding as well)
Most important hint: make sure the group is having fun. Observe the children's program a day or two before putting your child in the group (if possible). Are the children in the group unhappy and crying (continuously), or are they happy and smiling and laughing. That will tell you whether or not to use that particular instructor.