For the intermediate like yours truly, fear comes in different places from different kinds of degrees of difficulty.
1. First is the no-fall zone areas, where a fall would lead to genuine pain, and further catastrophic damage to body and mind. Note sometimes this is perception - incline of slope way steeper than anything one has experienced (say 50degs), where others might ski it fine but one has just crossed the threshold of one's own zone of perceived ability. Witness on this thread the pictures:
The pics are originally from the web (WB 3D and the guide to WB), to depict the sheer steepness of the Sapphire Chutes on the backside of Spanky's Ladder. Many ski it, but I assure you that is about 1500ft of sheer vertical which holds snow, i.e. the limit of vertical holding snow. I froze twice, and did the worst thing, hugged the mountain,) but did not have to go far, since you just have to reach your inside hand in about a 3/4 of a foot and your hand is on the mountainside) while descending on Sapphire Chute.
2. Cornices very high on a mountain while entering a steep bowl or chute, that jump-off necessary makes many a person faint of heart, including this one, rather enter from a "sidecut/lane" into the bowl, or chute.
3. Steepness is not the only factor, it's the length of the steep section, i.e. the longer and higher, i.e. the altitude of the starting point of the slope and the length of that terrain are definitely genuine intimidation accelerators, i.e pulse-begins-to-quicken when one looks over the top and realizes its a very, very long way down! For instance, in some of the conditions we saw, at Snowbasin in 2011 in January, slopes were chiselled rock-hard-ice, and one knew on the men's and women's downhill, one slip and that slide did not stop till the slope flattened out.
4. Combine long steeps, with ice-frozen conditions and extreme flat light conditions, fear is the key here, and staying calm becomes paramount.
5. Mandatory air entries, on high, again, not happy camper here when that is on the menu.
6. Cliffs over the side, or sharp ridgelines when one is traversing where one mis-step and well....
Just some thoughts on where this humbled intermediate gets intimidated.