Reducing evaluative bias from selection and development solutions

Dr Stewart Desson, CEO, Lumina Learning

What is evaluative bias?

Evaluative bias is where we value one thing more than the other. Typically, Extraversion is valued a bit more than Introversion. Typically, emotional stability is highly prized and the opposite, being a bit emotive, is less so.

Evaluative bias is not a new issue. Peabody researched this in 1967, but it’s not really been acted upon or taken too much notice of. We are working to address this issue which we feel is a bias. We don’t want to impose a bias that some traits are good and some traits are bad.

Why do we have evaluative bias in psychometrics?

We’ve found that the statistical techniques that have been employed in psychometrics over the last 30 years can unwittingly introduce biases if we’re not very careful. So, for example, if we gather lots of adjectives to describe the human condition, and we don’t really pay attention to what they mean, we just get everyone to assess themselves and we undertake a correlational analysis and factor analysis, it will start to segment yes into extrovert, introvert and so on, that does happen, but it will also start to put all the positive words in at one end, and all the negative words in at the other. So that’s why we sometimes get a bias where extroversion is seen much more positively and introversion typically, in these algorithms, comes out much more negatively. That’s a form of subtle bias that we at Lumina Learning are wanting to address.

In psychometrics, evaluative bias has a negative impact on:

• user validity
• construct validity

Why research evaluative bias?

In 2017, I completed a PhD around minimising evaluative bias in psychometric tools. This topic was of interest to me because of the bias prevalent in many current personality assessment tools. The research helps contribute to organisational psychology. Identifying and minimising evaluative bias helps to enhance interpersonal relationships and leads to more effective organisations through better developed leaders. This research is also very timely. Changing culture puts more focus on the need to avoid bias in the workplace and the need to value “deep diversity”.

So, how can we address evaluative bias in our psychometrics?

We can address evaluative bias in our psychometrics by not going with the traditional statistical approaches and by measuring both ends of the Big5 polarities both positively and constructively. Measuring both ends is very important as averages are not always very useful. If you stick your head in the oven and your feet in the freezer, on average you’ll be comfortable!

For example, we measure Extraversion both positively and constructively, as well as in an overplayed exaggerated form. So, you could be chatty and friendly, but you could also be a bit steamrolling at times. It’s the same for Introversion. We like to measure Introversion in a very positive way – you listen first before you speak – and then we could measure it in a not so helpful form, you know maybe I’m a little bit passive and don’t speak up enough and so on. So, we’re trying to address some of the bias issues that are present in many psychometrics because of the statistical techniques that have been employed.

Does Lumina Spark tackle evaluative bias?

Thirty to forty years ago, people said your personality was fixed, static, maybe you just need five high-level things and that’s it. But it turns out it’s not as simple as that – we have different personas, we change as we age, we change on different days, and we can in fact read a situation, tune-up and tune-down – there’s a dynamic. And essentially Lumina Spark is a dynamic way of looking at personality.

Lumina Spark:

1. Measures both ends of the Big Five polarities
2. Measures personality both positively and constructively
3. Values diversity
4. Reduces evaluative bias

In conclusion

1. Evaluative bias is (increasingly) an organisational issue.
2. The issue is addressed by measuring both ends of the Big5 dimensions.
3. Conceptualising constructive traits as the more extreme ends of the Big Five (“too much of a good thing”):
   a) Helps explain enablers and blockers to performance at work
   b) Avoids the risk of pathologising people
4. Lumina Spark’s approach does reduce evaluative bias. It enables users to crack open their Big Five dimensions and see their positive and constructive traits at both ends of the polarities.

Want to discover more about my PhD and how we’re minimising evaluative bias at Lumina Learning? Watch my presentation here or view the slides here.