I've owned a number of iPhones, currently have a 6. You don't need gloves off always. The volume down button starts it going, but you do have to take your gloves off a lot. Yes liners with things on the fingers for touch screens work, but on a cold day, your hands get awfully cold even with liners on.
Zooming is not great while filming, you can do it, but a point and shoot with a little lever/wheel on the start button is a lot easier, and zooming while shooting skiing is important.
My Panasonic Lumix is a lot easier to use and the zooming much better for skiing.
Also giving someone else your $650 phone to video you, and having them not know how to use it is an issue. I've gotten numerous videos with nothing on them, or facing the wrong way, or a still, or an "I thought it was running."
I love my iPhone, don't get me wrong, it's the most amazing device I've ever owned - but carrying a small camera that takes video and has a zoom wheel and that you can use with gloves is to be considered.
Finally the battery on my iPhone dies quickly in the cold.