I've done it twice now. My wife Lainie and I both skied ESA Aspen in 2007 and 2008 and brought along our little tyke Jacob who is almost 3. He skied with the ski school/day care this year. Snowmass has the best daycare/skiing arrangements I've come across in the quality of staff, facilities and ease of access.
There are some adjustments. It wouldn't work for my 16 year old, so I solved that by spending a week with him at Alta/Snowbird during his spring break. Also, at the ESA I had to cut from class to pick up my little guy a few times as did Lainie - we flip-flopped. Neither could we make all of the post skiing presentations and stayed at separate accommodations from the majority of the ESA participants. But we both thoroughly enjoyed the ESA.
As nyc mentioned, my wife and I skied together both before and after the ESA; we made a week of it.
From a family perspective, Snowmass is hard to beat since most condos are ski in, ski out and has a great shuttle service.