Over the course of my career, I’ve conducted hundreds of interviews. I’d like to think I’m a good judge of character and that what I learn in interviews helps me make good hiring decisions. But, if I’m honest with myself, I have to admit I’m biased. I like confident extroverts who read a lot and come off as curious. In other words, I’m biased towards people who tend to be sort of like me. Of course, this is very common. Here’s a super interesting IFOD on this point: Implicit Egotism.

Also, I probably have biases of which I’m not aware. And – how valid are first impressions? I thought one of my now best friends was sort of an asshole when I first met him then I got to know him and really liked him.

Research validates my skepticism. Interviews, it turns out, are nearly useless. At least in the unstructured way they are usually conducted.

What Research Says

There have been hundreds of research studies on the effectiveness of interviews. The studies find that at best unstructured interviews have a slight predictive benefit but the majority of studies find that interviews confer no benefit in selecting qualified candidates and are often counterproductive.

Here are some examples from research:

EXAMPLE ONE: In 1979 due to a late legislative change, 50 formerly rejected applicants were admitted to the University of Texas-Houston medical school. These 50 applicants were rejected on the basis of their interview performance as compared with their admitted peers. What happened when these poor interviewers were then admitted? Over their medical school careers their performance was indistinguishable from the other students. This result is consistent with other studies which have found that medical school admissions interviews add no value. Specifically, “Meta-analyses of over 150 studies demonstrate that mechanical/formula-based selection decisions produce better results than decisions made with human judgement.” 

EXAMPLE TWO: In a fascinating study, researchers had students at Carnegie Mellon interview other students and then predict their GPA for the upcoming semester. In addition to the interview, the students had access to the student’s course schedule and their past GPA. As a control, the student predictors were given the past GPA and course schedule for students they didn’t meet and asked to make predictions without the benefit of an interview.

What happened? The GPA predictions were significantly more accurate for the control group of non-interviewed students. It turns out that past GPA is the best predictor of future GPA and the interviews only distracted from that solid information. The interviews were totally counterproductive.

Notably, when a new batch of student volunteers were told they were to predict upcoming GPAs for other students, were told conclusions of the previous study (that the interviews were counterproductive), and asked whether they’d like to conduct interviews, the students overwhelming chose to conduct interviews. Thus, even when faced with research that says otherwise we tend to think we are special and can still middle people on the basis of an interview.

EXAMPLE THREE: When Daniel Kahneman, the Nobel Prize winning psychologist, was a young officer in the Israeli Army he was asked to evaluate the procedure used to determine how well recruits would do in the army. The existing procedure used a combination of tests as well as interviews by trained interviewers. Kahneman determined that the interviews were “almost useless for predicting the future success of recruits.” Why? According to Kahneman, “conducting an interview is likely to diminish the accuracy of a selection procedure . . . because interviewers are overconfident in their intuitions, . . . will assign too much weight to their personal impressions and too little weight to other sources of information, lowering validity.” So, what did he do to improve the process? More on that below.

Why are Interviews so Ineffective?

• Dilution Effect. There are characteristics which correlate highly with job performance. These factors vary by job. For example, factors suggesting success for an accountant might include numerical fluency, attention to detail, good time management, excellent organizational skills, etc. Focus on these important factors likely will be diluted by other impressions from an interview. Maybe the candidate is wearing a tie that doesn’t match, or is quiet, or talks about their four cats for too much of the interview. These things are likely distractions that dilute the focus on the factors that do correlate with suitability for the job.

• We are biased towards attractive people. A classic study on this point from the University if Minnesota proves this point perfectly. Men were told they were going to conduct a phone interview with a woman. Some were shown an attractive picture and some were shown an unattractive picture. Of course, the women on the phone were not those in the picture – they were randomly chosen. What happened? The “attractive” women were rated more highly than the “unattractive” ones by the interviewers. Further, when independent third party volunteers were asked to listen to the phone calls without seeing the pictures, the third parties ranked the attractive women higher as well. What? How did that happen? According to the book The Best Place to Work, “when the men thought they were speaking to an attractive woman, they behaved in ways that brought out their partner’s best qualities. They acted more friendly and demonstrated considerably more warmth.” This isn’t crazy – there is an evolutionary reason we prefer beauty. But, attractiveness just isn’t relevant to the performance of most jobs. Related IFOD on what is beauty and why we prefer it here.

• We perceive taller people (especially when judging males) to have greater leadership ability.

• People with lower voices are associated with greater “strength, integrity and leadership.”

• When judging someone for the first time we tend to focus primarily on two factors: (1) their warmth (e.g. friendliness) and then (2) their perceived competence. Good criteria. What’s the problem? It turns out that when someone projects a lot of warmth on an initial meeting we infer that they are low on competence. Thus, we subconsciously associate warmth and competence as if they are inversely related. Of course, warmth and competence are independent and one can be both warm and competent as well as cold and incompetent. Thus, first impressions are often be subconsciously flawed.

• The problem is not just the interviewers. One study found that just over 80% of interviewees lie (or “bend the truth”) during job interviews.

• The structure of interviews is not relevant to job performance. Think about it. Your interviewee comes into a conference room and is nervous and is thinking about every word they say. You spend 30-60 minutes talking with this person, usually in a mostly unstructured fashion peppering them with questions. How does performance in this arena relate to what they are going to do day-to-day in the their jobs?

What to Do?

While interviews are ineffective and often counterproductive, it also doesn’t make sense to hire someone without meeting them. The key is to be smarter about how we go about evaluating candidates. Based on research, here’s what I’ve found are best practices:

1. Create structured interviews. Daniel Kahneman found that using a series of standard questions, asked in order, that were relevant to factors related to success greatly improved the interview process. Research seems to be unanimous that unstructured interviews are not useful, and are usually counter-productive while structured interviews are more effective. Use multiple interviewers and have each person responsible for evaluating one or a few relevant criteria. Use the same questions from candidate to candidate.
2. Use behavioral interviewing techniques. This type of interviewing uncovers a candidate’s past behavior in various situations. For instance, asking a candidate “tell me about a time you dealt with an unreasonable or difficult client” is much more illuminating than asking “do you deal well with difficult clients?” This is because past behavior is very predictive of future behavior and takes self-objectivity out of the equation. They key is to ask behavioral questions that relates to behaviors correlated to job success.
3. Scoring. Have interviewers take notes and score the interviewee based on criteria relevant to job performance. This reduces subjective bias.
4. Ask applicants to complete assignments related to the job they’ll be doing. If you are interviewing for a chef, have them cook some meals. If interviewing for an investment analyst, have them analyze some securities for you. Etc.
5. Improve your pool of candidates. In the book The Best Place to Work the author suggests improving the flow of referrals from existing employees. He notes that “studies indicate that on the whole, referred hires outperform those who get theirs jobs through more formal channels.” Also, seeking out potential candidates who are not job searching greatly increases the breadth and quality of the candidate pool. Use of LinkedIn and/or spending time seeking out qualified candidates not job hunting using other means often pays dividends.
6. Score the candidate based on criteria from outside of the interview. Consider using cognitive and personality tests to learn about candidates separate from the interview. Grade the relevance of their educational and work experience.