Typical interviews are less predictive than we think

Typical interviews are less predictive than we think

1080 651 Simon Desjardins

Interviewing effectively is surprisingly tough. It’s a process that sometimes seems straightforward and yet often leaves us feeling like we haven’t quite gotten the clarity we were hoping for out of an hour-long session, or that we’ve simply made a gut decision.

The main reason for this feeling lies in the unstructured nature of interviews, which makes it much harder to reach a conclusion about whether the candidate would actually perform well or not.

In this series, we will explore some central challenges we face in the interview process, and highlight best practices and tools to make immediate improvements.

The statistics about interviews are both counterintuitive and somewhat alarming: the chances that an unstructured interview will accurately predict a candidate’s performance is less than 25 percent. You’d actually be better off flipping a coin, and would have saved several hours in the process!

Anatomy of an unstructured interview

  • Candidates are asked different questions by each interviewer.
  • Questions are not necessarily linked to key competencies required to do the job.
  • Candidates are not assessed using a standardized rating scale.
  • Interviewers haven’t aligned on acceptable answers beforehand.

With dozens of studies across multiple geographies and timelines showing them to be one of the worst ways to predict on-the-job performance, the evidence against using unstructured interviews is overwhelming. Yet we continue to rely on them almost exclusively to make hiring decisions. What’s going on?

Limiting the effects of bias

Bias plays a huge role in how we rate candidates in an interview setting. Humans are naturally and strongly predispositioned to favour people who are like them, which is a hazard when our objective is to build diversity on our teams.

Compounding this problem is our susceptibility to “first impression” bias, where we consistently end up asking easier questions of people who form a strong first impression, and harder questions for those who don’t. In their now-famous study, Tricia Pricket and Neha Gada-Jain showed how snap judgements made in the first 10 seconds of an interview could predict the outcome of that interview.

Mere awareness of these biases does little to counteract them, even for experienced interviewers. Asking candidates to complete a competency-based assessment process before the candidate reaches the interview stage adds a layer of objectivity to the screening process and can help put interview results in context.

Linking interview questions to competency requirements

Great interviews start with the interviewers aligning on what competencies and other requirements will actually drive performance in a given role. It may sound obvious, yet we often see a majority of an interview being spent asking questions without a clear competency in mind.

Brain teasers, for example, are a perennial favourite, though they have been shown (most conclusively by Google’s HR team) to have no correlation to performance. By a similar token, academic scores (and by extension — questions about them) have equally little bearing as predictors of performance, unless we’re hiring people immediately after they graduate.

Asking questions that actually force a candidate to reveal a key trait can be risky without preparation and thought. For instance, if we wanted to understand a candidate’s “motivation to join,” we might be tempted to ask the basic question, “Why do you want to join our company?” This question, however, is easily answered by a clever candidate who has done their research. A less obvious but more revealing question, such as “What preparation did you do in the time between when this interview was scheduled and today?” might give us a much more meaningful data point on the same subject.

Interviews and the Stanford prison experiment

Back in 1971, Stanford University psychologist Philip Zimbardo led a set of experiments that changed what we know about human behaviour. Zimbardo arbitrarily assigned participants to play the role of “prisoner” and “guard” in a role play exercise, and the inhumane behavior of the “guards” revealed the extreme psychological effects of perceived power. The results were replicated many times across multiple countries and cultures.

Interview rooms aren’t prison cells, of course, though much of what was learned in those studies applies to this context. Given that employers have a job to offer, we sometimes assume a position of authority in relation to the candidate, and act accordingly, without realising it. Imagine, however, that we’re interviewing a top performer who has multiple employment options. In this situation, we’re being interviewed as much as we’re interviewing. Leaving five minutes at the end to answer a candidate’s questions won’t be enough to properly address concerns and communicate why the candidate should join us.

Interviews driving improved performance

Admittedly, interviews tend to be high stakes environments for both parties. We’re under pressure to make the perfect hire. The candidate is nervous. Scepticism, hope, and bias are at risk of permeating every exchange. If you leave time to “sell” the candidate at the end, we’re down to 40 minutes at most to ask five to eight questions. Given the exponential impact that high performers have on organisations, these 40 minutes are crucial.

Putting more thought and structure to that time will separate you from the vast majority of other hiring companies, including your competitors.

If unstructured interviews don’t work, then what’s the answer? In the next article in this series, we outline the basics of the “structured interview” and when combined with competency-based assessments, will save time and significantly improve the outcomes of the interview process.

About Shortlist Advisory

Shortlist Insights helps companies build capacity to improve how they recruit and manage talent. We combine best practices from industry experts, research, and our experience to deliver practical and tested solutions and thought leadership. Ultimately, we help our clients build a competitive people advantage.