Originally Posted by Trekchick
To the point about bias' - That may be true to some point but I also hear from other testers, "I don't usually like (insert brand here) but this ski really wow'd me" or "I usually like (insert brand here) but I can't warm up to this one"
On a personal note, I (as you know) am a big fan of Blizzard skis these days, but in the past I never liked the Crush, a ski for which @segbrown is a big fan.
When I tested Head skis in the past I found them to be uninspiring and quite unforgettable, but I tested the Joy line up of skis and was "WOW!"
Funny thing is, I've been a big fan of Kastle for 6+ years but I recognize some skis in their line up that don't fit my skiing or my physiology.
I think a good tester can set aside the bias' and be open minded if he/she is worth her/his salt.
Interesting points. I hear what you're saying, but the road to hell is paved with psychology studies showing that none of us can truly set aside bias, because as scientists use the word, it's mostly unconscious. Just a systematic direction of error, like tending to tilt your head slightly to one side when you read a ruler, so that every time you're a mm above what other folks might see. Not conscious, not deliberate. I doubt testers set out to commit fraud, as in "Ah, I know this brand and I like it, and they give me $$, so I'm going to make sure it has a higher score." With subjective assessments, more often, the drill is simply a very slight warming or positive expectation about a particular outcome: "Ah, he's done good stuff in the past, I'm going to enjoy his argument, I bet." Or, "I tend to like Kastles, this should be fun." Doesn't mean we can't ski a Head that we are amazed about, or be disappointed by a Kastle, more a sort of overall inclination to give Kastles an average benefit of the doubt if a score is borderline. Cuz we know they're cool skis. Or we use criteria that Kastles are strong at to evaluate other skis, so they'll always be a beat below the "gold standard."
This is one reason that Ski Canada lists its testers, including not only size and qualifications, but who if anyone sponsors them and what their personal skis are. Surprise, testers who are sponsored tend to list their sponsors' skis as 1) among their personal skis and 2) as among their 3 favorites in the test. Unsponsored testers tend to list new models of their personal skis as among the top 3. (In all cases, not necessarily the top choice. Not that glaring.
You could say that since they already like them enough to ski them all the time, they'll probably also like the new models. Or you could say, brand loyalty. Or you could say that's a nice example of intraobserver behavioral bias...
Which is why, contrary to popular belief, scientists don't try to eliminate bias. Rather, the trick is to know what it is, and how it might affect outcomes, and declare it up front. Measure it, even. (Called a reliability test, which you'll never ever see in a ski test). And why the conflict of interest issues are so big in medical journals right now.