The Last Question

So, we need to use multiple methods. But, in which order?

Say that you apply for a Sales Manager job at the Acme Corporation. First, they conduct a phone interview. Then, they call you for a face-to-face interview. After that, you visit for a whole day of testing: aptitude tests, situational tests, personality tests, the whole shebang. The Friday after that, you have an on-site interview with your prospective manager. Ten days later, you have an evening interview with his manager. More than a month has passed since you applied, and you’re five rounds in. Annoying, since you’re currently employed and need to make excuses to go for each interview or test. Finally, they send you an email stating they want to make you an offer. Again, you come to the shiny Acme Corporation office for a final talk with both managers. They offer you a salary you are satisfied with—great. You wipe the sweat from your forehead and relax into your chair.

Just one more question, your would-be future boss asks, “Tu español es bueno, ¿no?”

“Erm. Sorry, what?” you say, flustered.

“You don’t speak Spanish?” he replies. “Oh, maybe we forgot to mention it, but you’ll be working a lot with our partners in Mexico. So, we need someone fluent in Spanish. Sorry.”

What would you do in this situation? I would probably take the nearest blunt object and launch it at their heads. How could anyone forget such a simple requirement as “fluent in Spanish”? All those afternoons wasted, for both you and them, just because they forgot to ask one question.

This story would be funnier if a similar thing didn’t happen to me, both as a candidate and as an interviewer. It is a typical example of a minimum requirement which takes less than a minute to check, yet it wasn’t checked until the very end of the hiring process.

How you order your screening process makes all the difference.

Reward vs. Effort Ratio

When I started doing screening, I didn’t understand this. At the beginning of the process, I did both all the things that were traditionally done first and the things that were important to me. Everyone else began by screening résumés, so I did that too. Meeting a candidate for an interview was important to me, so that was a logical second step. I understand, now, that I was completely wrong in the ordering.

A screening process is a kind of funnel that collects information at each step which can predict future behavior. But, collecting information has a cost in both time and money. In other words:

Screening_Efficiency = Reward / Effort

Accordingly, the screening process should be reordered, so that the most efficient methods come first. Higher efficiency methods give the most valid information to make a decision, saving time for both you and the candidate.

In the Acme example, an on-site interview gives more information than checking Spanish fluency, but an interview takes hours of managers’ and candidates’ time. Checking Spanish fluency gives less information than an on-site interview, but it requires only a one-minute phone call to know for sure whether you can reject them. It is vastly more efficient.

Why the Traditional Model Is Not Efficient

Let’s examine effort and reward in the traditional screening process for both candidates and employers. Here’s how the Acme Corporation process for an average candidate might look:

As you can see, a huge amount of time is invested in each application by both the company and candidate.

The problem occurs when you look at the last column. The point of screening is to collect the maximum reward (information) for the least effort (time). The first steps don’t give much useful information at all. A résumé tells us years of education or interests (validity of 0.1) and job experience (validity of 0.18). A short, fifteen-minute phone screening also has low validity. A company must invest more than five hours to get their first piece of high-validity information from the knowledge test. So, why structure things this way around?

The traditional approach is popular because, first, most people don’t question the way things have always been done. Secondly, because people are uncomfortable asking tough questions. Most companies spend the first few hours with “soft” topics (conversation and personality tests) before they go to the “hard” topics (testing knowledge and asking direct questions).

This makes sense for dating, because we often enjoy the dating process. But, it doesn’t make sense for hiring —because both an interviewer and a candidate would rather be elsewhere. Being direct saves effort for both.

If a candidate passes the entire process, it takes 7 hours and 15 minutes of company time. Let’s look at how much time could we save by eliminating them earlier:

Eliminating a candidate from an application form is 87 times more time efficient (5 minutes vs 7 hours 15 minutes), and that’s before you multiply this again by the number of candidates. If you have 20 candidates and an average applicant takes five hours to screen, failing after the knowledge test, the total effort is a huge 100 hours.

Surprisingly, real-world companies cover the entire spectrum of efficiency. For example, one government agency which I know of would put every applicant through the almost full procedure, so they would be close to 1x. Needless to say, they have a large HR department. At the other end of the spectrum, some high-tech companies have automated screening, so that they can screen thousands of candidates with very little effort.

What is the efficiency of your company? How much, on average, does it cost you (in time and money) to eliminate a candidate?

Recruiters often say that they are very efficient, because they eliminate the majority of candidates from their résumé alone. But, if 90% of candidates are eliminated based on low-validity criteria, then that process can be summarized with this joke:

A recruiter splits a stack of job applications, and throws a half of them in the garbage. “What are you doing?!” asks his coworker. “Well,” answers the recruiter, “we don’t want unlucky people in our company, do we?”

Ways to Reduce Effort & Increase Reward

There is one ingenious idea rarely mentioned in screening resources: find a person who knows the candidate the best, and convince them to do a screening—for free. That person is the candidate themself, and the method is candidate self-selection. Candidates don’t want to be failures: they want to succeed in their job. If your job description indicates that they are probably going to fail if they were hired, they’re not going to apply.

When advertising normal products, you want to sell to as many people as possible. But, when advertising jobs, you want to “sell” it to the right people for the job—and scare away everybody else.

I realized this the first time that my company was advertising a managerial position. Our previous job ad was for a developer and we had 30 applicants. But, when we needed a manager, which was a more demanding job, more than 140 people applied. The problem was that our developer job ad was very specific: we listed languages and technologies which candidates needed to know. The manager’s job ad was more general, and everybody thinks they would be a great manager. It didn’t have specific criteria that candidates could use for self-elimination.

Whenever you have a job ad that is too general, the quality of candidates will decrease. Therefore, you must always provide a realistic picture of what the job actually entails, and your selection criteria for getting it. If your job requires frequent weekend trips, say so. If the necessary criteria is at least two years of experience managing five people, say that loudly, in ALL CAPS if you must. It’s saving everyone’s time.

If the job description is too unspecific, you will end up with hundreds of candidates but run the risk of your hire quitting after six months because the job was “not what they were expecting.”

Sounds obvious, but I used to make the mistake of leaving out one crucial part of the job description, just because other companies omitted it too—the salary range. While I knew exactly what salary range my company could afford, salary negotiations are difficult and awkward, so I left them for the end. As a result I spent hours and hours interviewing some senior candidates, only to discover that—aside from a unforeseen lottery win—I could not afford them. I wasted their and my time.

The second way to decrease effort significantly is to employ the magic of automated testing. Here, the computer can screen hundreds of candidates for near-zero cost, once we’ve invested the one-time effort of developing a good automated test.

The third way to decrease effort is to divide testing into two parts—short tests early in the screening process and longer tests later. Most candidates will fail the first test anyway, so we will save time by not doing longer, unnecessary tests with them. This also saves time for candidates, as they only need to take the longer test if they are qualified.

To increase the reward, we need to use high-validity methods as early as possible. Every method in the screening process asks the applicant to answer questions. These can be as simple as the “Name” field on the application form, as complex as answering a mathematical problem in a structured interview, or as straightforward as stating whether they will accept a starting salary of $65,000. Our rewards are the answers to these questions. They tell us whether to send the applicant further down the funnel or back home.

Every method, whether it has high or low validity, has one thing in common—it’s only as good as the questions asked within it. So, the next chapter is dedicated solely to the art of asking good questions.