For Organizations, Interviews

The Myth of the Magic Question


Jay Gould, cofounder and CEO of Yashi, promises in Fast Company that he’s discovered the “Best Interview Question”. He asks this question after conducting 2-3 other interviews that have confirmed the general skills and strengths. At this point he is evaluating fit and motivation. Like many claims of magically powerful shortcuts, this one’s wrong. In fact, Gould’s question puts even more unfair pressure on job candidates.

Most “great” (read unchallenged and mostly arbitrary) interview questions lack a consideration of context and therefore create inequities in evaluation. They require the candidate to interpret what the interviewer is looking for and how the interviewer values different answers. In this case, Gould’s magic question is “Why should I not hire you?”

He explains that the cleverness of this question is based on the assumptions that:

  • The candidate likely hasn’t prepared an answer (true, which makes #3 below likely false)
  • It gets people to be honest about their mis-fit with the role (perhaps, but likely inappropriately. How “good” must their mis-fit answer be to be judged a good fit for the organization?)
  • Offers a fair chance for both slick, salespersons and introverts alike (it largely does the opposite by creating more pressure for spontaneous social interpretation and delivery)

Gould says that you learn everything you need to about the candidate when you look them in the eye and ask this question. If the candidate can’t–or refuses to–answer this question Gould doesn’t hire them. But let’s unpack how unfair (and therefore ineffective) this question is.

A QUESTION OF CONTEXT

My primary complaint with presenting this as advice is that there’s no indication of which information is provided to candidates earlier in the interview process. The question, “Why should I not hire you?”, requires the candidate to have a detailed understanding of the role, company, culture, leadership style, and of course herself. In asking this question, Gould is asking candidates to do an extremely complex assessment of the situation and themselves and deliver a coherent answer quickly. Gould even says he uses any delay as an indication of deception or misrepresentation–it’s far more likely attributable to the overwhelming analytical challenge. All of Gould’s algorithmic interpretations mean that a candidate’s answer must land in the Goldilocks zone–not too negative that it reveals a true disqualification, but “real” enough to prove “authenticity.” Yikes.

All of his algorithmic interpretations mean that a candidate’s answer must land in the Goldilocks zone–not too negative that it reveals a true disqualification, but “real” enough to prove “authenticity.” Yikes.

If Gould and his team were hyper-transparent about the culture–talking more and asking less–there is a chance that this question might provide some kind of valid information.  Candidates can only use the information they have been given along with their projections about the organization. If they have little of the former, answers will be driven by the latter–which opens the door for unintended discrimination. But even with high levels of sharing from the organization, candidates who are deeply honest and have a tendency to transparency and not manipulating others (perhaps candidates Gould may want to hire), will use the limited information they have and filter their bad qualities through it. They’ll overshare. They’ll likely give awkwardly negative answers. To see how unfair and ineffective this question is consider using it on a date.

The parallels are remarkable in terms of the stakes in play, the limited information, and brief evaluation window.

THE 3rd DATE TEST

I like to evaluate interview strategies in hiring by comparing them to interview strategies in dating. The parallels are remarkable in terms of the stakes in play, the limited information, and brief evaluation window. They’re also similar with respect to the outsized influence of intangibles and biases/heusitics.

Imagine, you’re on a third date with someone. The first two dates went well. You’re feeling pretty good so far on the third; things are clicking. Then toward the end of dinner you ask: “So, before we move forward, why should I not be in a long-term relationship with you?” Here’s what happens next.

The person doesn’t answer immediately. Well, you may then judge them to be hiding something. Do they stumble through their answer? Are they stunned and even appear to resist answering? Well then, you may conclude that they’re definitely hiding something and wouldn’t be a good match for you and your honest cleverness. You pay the check (only decent of course given the prospects of that poor person) and walk out briskly, your SparkNotes of Atlas Shrugged tucked under your arm, confident that you’ve avoided a terrible relationship.

From society’s perspective this would never fly. If someone observed this exchange from a nearby table, they’d call the person who asked this of their dinner partner an a**hole. We would never conclude this person was effectively and humanely evaluating partners.

AN HONEST ANSWER

What might an honest, transparent answer to this question look like? In the dating scenario it might go something like:

“Um, well I was hoping to find a good partner to explore the world with and with whom I could develop a strong system of relating, but since I haven’t found that yet and my other 5 long term relationships didn’t work out, I guess you shouldn’t be with me because I haven’t figured it out yet. I also don’t know you well enough to help you with all of your problems and haven’t learned of a healthy way to risk sharing my deepest insecurities with others. I also sometimes pass gas in public places, and if I’m being honest I think it’s healthy and did a few minutes ago. I guess those could be the main reasons…but I don’t really know you well enough to give much more detail.”

Either this answer would seem endearing to you and draw you toward them (a result of your own relational biases), or–because it’s not an easy fit and the person doesn’t immediately know what you need and what you don’t– you’d dismiss them and go back to OKCupid.

A BETTER PRINCIPLE

In the workplace–especially at the hiring moment, there is a tremendous power difference (the executives are the ones with the jobs to give out). That power isn’t challenge-able in the same way as it might be in the dating arena (where perhaps we can imagine a retort of “Well, why shouldn’t I be with YOU!?). In the hiring context, the power imbalance allows executives to mistake their own laziness for cleverness.

I see this mistaken pursuit of stronger interviewing signals with so many executives and it’s something I’m helping a few of them improve.

But there’s a better principle to deploy. With dating as in hiring, good decisions must start with the decider risking honest transparency first. Without that shared knowledge foundation (especially from the person with power), the person on the other side won’t have the information they need to describe a potential match from their perspective.

Given the power dynamic, Gould could first tell the candidate why they shouldn’t work at Yashi (why Yashi is bad for them) and mean it. I’d be very curious to hear how that would sound–if done well I think it could send good signals to candidates and yield better decision making for both parties via that established trust. But I believe the company has to lead. Companies are currently awful at this and it’s ripe for improvement.

Without transparency executives are relying on lightening to strike. And in doing so, they rely on arbitrariness rather than rigor.

________

Call for Collaborators

Has your organization been dissatisfied with its hiring outcomes? I’m looking for a few organizations with which to co-create something better. Email me at team (at) theleancareer.com.