How about a third option: Make a primary residence on or near a world class ski resort .
The two-choice OP question is a no-brainer for me. Unless there's strong likelihood that the real estate market at said nearby resort is going to skyrocket, I'd go with the travel. A second home could become more a project and money pit than a vacation. Plus you're stuck skiing the same small mountain instead of going to the places most skiers really want to ski.
You can take a lot of nice ski vacations for the price of a ski home - and by choosing that route, you can even justify splurges like heli-skiing.
After owning a second "home" near a "small" hill, I did the third option (sort of - see below). A portable skill set and no kids helps.
The second home worked well, and we enjoyed it. It may not really be what the OP was asking, though. It was a small studio condo (with a handy-dandy hot tub and pool, and covered parking - important in a place where it snows a lot), so not exactly a home in the conventional sense. The "small" hill was this little place owned by the City of Denver - Winter Park, Colorado. It was two hours from where we lived in Fort Collins.
We had it in the rental program briefly, until some short-term renters pulled the new towel bars off the wall and broke some of the new floor tiles. We had done some nice reno, and the management company charged the highest rate for the place, but they weren't interested in charging the renters for the damage. So I pulled it from the rental program and happily used it 70-75 days during the ski season and another 20 days or so in the summer.
Now I live well under an hour from two local resorts. Whether either one of them is "world class" is debatable, but they're certainly decent enough to attract skiers from quite some distance, even though flying into here is challenging. I ski more powder every season than I did in 20 years at Winter Park. And there are many things to do around here in addition to skiing.
I've got the answer for you right here:
Good example. In that case would you rather pay US ~$1 million for nice house with five private slopes, one t-bar, and 250' vertical near Toronto OR three vacations (1-2 weeks each) at US $10,000 each vacation per year for the next 33 years?
This one, for me, is a no-brainer, and is sort of what I thought when I read the OP. Traveling with skis is a hassle, but 250' of vertical isn't particularly interesting, especially at that price. For $10,000 per week, I can go heli-skiing and still not have to submit my skis to airline baggage handling practices, since several major heli-ski operators are between 10 minutes and 2 hours from my house. If I had a million US dollars to burn, I'd do the heli option. Frequently.
For a little less, there's always cat skiing. Either is better than a money-pit house near a Mickey Mouse ski hill.
We might also note that ownership of that second house near the small hill may impact your ability to visit big hills because you're paying for the house. Except for convenience, I don't see the value - for me, anyway.
If you really want to ski a lot, look for employment in Denver or Salt Lake City. Then, if you feel the need for a second house within two miles of the hill, at least it's a big hill!