Role-Based Auditions: A Case Study About Improving the Interview Process

The Challenges

Our clients—the heads of HR, Talent Acquisition, and Sales—recently tripled the size of the Sales organization during a two-year hiring surge. Halfway through that journey, they began to see a disturbing trend. More than 40% of their newly hired sales representatives resigned or were terminated within 6 months of hire, mostly for reasons related to under-performance. This put tremendous strain on the organization. In too many cases, Reps were hired for vacant territories, attended weeks of training, were onboarded by their manager, underperformed for months until they exited, then the organization had to reincur those costs only to see the pattern repeated one or two more times. We were hired to investigate and identify the root causes of these poor hiring numbers.

#1: Unclear Hiring Criteria

Our analysis identified the primary root cause: a lack of alignment on hiring criteria among recruiters, hiring managers, and the senior Sales leaders. Hiring managers were asked to describe why they selected a particular candidate, and not only did their responses vary widely, but the attributes they prioritized did not align with the ones their Sales leaders prioritized, nor did they align with the attributes associated with the most successful hires. In other words, many hiring managers were focusing on the wrong factors during the interview process.

#2: Lack of Support for Selection Decisions

Despite their interview process having multiple stages, with the hiring managers having several opportunities to meet with their candidates, the hiring managers had difficulty recalling any specifics discussed during the interview process that would back up their selection decision. For example, a hiring manager might say that they selected a candidate because of their drive, but when asked to provide examples of why they felt the candidate displayed drive, they could not recall any. “If you spend hours with a candidate over multiple meetings, one would think you’d remember something,” complained the head of Sales.

#3: Poorly Executed Interviews

All hiring managers had been trained to use the STAR method of interviewing, which involves asking a “Tell me about a time when you…” question that focuses on a specific Situation, then following up to learn what responsibilities the candidate was Tasked to complete, what Actions they took, and what Results they achieved. However, we found that the least successful hiring managers were instead asking more general, “Tell me about yourself” types of questions.

The Solution Architecture

Step 1: Hiring Criteria

We presented our analysis findings and recommendations. One of the first key steps was to get alignment on the hiring criteria among the recruiters, hiring managers, and sales executives: what attributes should they prioritize and which attributes could they compromise on when making hiring decisions?

Based on the team’s input, the agreed-upon criteria revolved around a candidate’s drive for results, their communication skills, their business acumen, and their clinical acumen.

HIRING GUIDE.To help everyone understand, align on, and apply the criteria, we created a Hiring Guide that provided clear descriptions of each attribute, examples of what each attribute looked like a cross different candidate archetypes, sample questions the interviewers could use, what to look out for in the responses, and ways of probing deeper through follow-up questions.

The Hiring Guide was then incorporated into a workshop involving the hiring managers and senior sales leadership. This gave the whole team an opportunity to discuss, clarify, and align on the hiring criteria.

examples from Hiring Guide

CONTEXT MATTERS. While their hiring criteria were common to other sales roles, the contextual details gave each attribute a company-specific flavor. For example, drive. Sales reps needed to be able to influence the physician’s prescribing behavior, but that was only part of the challenge. Once a script was prescribed, the rep needed to ensure that the patient got access to the medication quickly. This pull-through sometimes involved days of persistent nudging. Even once the patient received access, the patient might have declined to use it or the need might have passed, so drive needed to include influence and persistence, as well as resilience.

COMPROMISE. There are no perfect candidates, so hiring always requires some degree of compromise, where the hiring manager is willing to accept less of some attributes to get more of other attributes. For example, one might be willing to give up some business acumen or clinical acumen to hire a candidate who has a very high drive for results. The hiring managers needed to be aligned on how they would evaluate those trade-offs and needed to understand how to manage the associated risks. For example, hiring someone with less clinical acumen may require spending more time with them, helping them learn the product.

Step 2: Candidate Profiles

One of our findings was that hiring managers felt they would benefit from being able to visualize what a strong candidate would look like on paper. We worked with Talent Acquisition to identify candidate archetypes, representing what good ones look like on paper while accounting for other factors like industry and job experience.

That work yielded 6 target profiles, ranging from low-experience recent graduates to high-experience industry insiders.

APPROACH. For each of the six target profiles, we provided guidance on how to approach the interview and included a set of recommended interview questions to use during the interview process.

For each of the six target profiles, we created a sample resume and recruiter comments, which the hiring manager would receive with their slate. We also provided some considerations, explaining how the attributes seen in the resume could potentially map back to the hiring criteria.

examples of Target Profiles

Step 3: Interview Questions

How can we ask better interview questions that get directly at the hiring criteria?

During our interviews, we asked hiring managers to describe how they prepare, conduct, and debrief after interviews, and we found that the lower performing ones did not prepare questions in advance, did not ask questions linked to their competency model, and did not ask questions using the STAR method (Situation-Task-Action-Results). In the Hiring Guide and in the workshop, we focused on getting all hiring managers to follow these best practices.

INTERVIEW PREPARATION. For best results, hiring managers should spend 10-15 minutes preparing for each interview. This involves reviewing the resume and any notes from the previous round, as well as deciding which questions to focus on during the interview.

SKILL-BASED QUESTIONS. In behavioral interviewing, the STAR model is often used with the intention of getting candidates to describe what they were thinking and doing in a given situation.

While the STAR questions focus on looking backwards “…at a situation when….”, other skill-based questions can ask the candidate about the present and future. For example, “Walk me through what a typical week looks like for you.” Or, “What skills are you hoping to improve this year? What are you doing to get better?”

examples of the Approach, Watch-Out, Assumptions, and Question notes for a particular profile

CHECK WATCH-OUTS. The interview questions selected should help the hiring manager delve into areas that are watch-outs, where they can either confirm or dismiss concerns about the candidate’s lack of aptitude in a particular area. For example, a sales rep with only a year or two of selling experience might seem too green on the surface, and yet maybe they learn quickly and have a predisposition for it, so it is vital that hiring managers give candidates a chance to show what they can do.

CHECK ASSUMPTIONS. Questions should also help the hiring manager check their assumptions to validate whether perceived aptitudes exist. For example, if a candidate has a number of sales awards on their resume, one might assume that they have strong selling skills. Still, the hiring manager should delve into those selling skills to make sure perceived aptitudes are backed with real skill.

Step 4: Interview Auditions

Another finding was that none of the lower-performing hiring managers and only a few of the higher-performing ones were incorporating role plays or any other form of auditioning into their interviews. This seemed like a missed opportunity, because there is no better way to gauge someone’s selling skills than to have them demonstrate those skills live..

We asked the hiring managers, “How do you know that your candidates can sell effectively?” Many of them responded that they gauged this ability based on years of relevant selling experience, sales awards, and the interpersonal skills that they demonstrated during the interview process. Awards and other resume entries can be embellished, and just because someone communicates well doesn’t necessarily translate into their being able to sell well..

We asked the hiring managers to start incorporating auditioning questions into their interviews, and we provided examples of what those questions could look like.

AUDITIONING EXAMPLES. Given how important selling aptitude was, all RSMs should have been incorporating some sort of sales audition into their interviews. That could simply involve giving the candidate a scenario, then roleplaying the conversation they would have with a healthcare provider.

• Their Product: That scenario could revolve around selling their previous product, something they presumably already know how to sell.
• Fictitious Product: It can also involve a fictitious product or an everyday product like hand soap or a pen, but that is less relevant to the kinds of sales conversations they will have in a healthcare setting.
• Your Product: An even more powerful method involves sending the candidate to your company’s website to learn about a product, then coming prepared to role-play the conversation at the next interview. This not only demonstrates their selling skills but also demonstrates how driven they are and how quickly they are able to learn about a new product and use that information in a sales conversation.
• Coachability: The hiring manager could take it a step further by providing the candidate with some feedback to see how well they react to coaching. Coachability is an often-overlooked factor that hiring managers often wish they had assessed better in the interviews.

Step 5: Interview Milestones

The RSMs were conducting multiple rounds of interviews, but one of our findings was that they approached each in the same way, often asking the same questions in a slightly different way. They were not delving deeper, and they were not checking their biases along the way. In fact, most of them reported that they made their decision after the first interview, and the rest of the touchpoints simply gave them opportunities to reinforce their initial assessment—the very definition of confirmation bias—and build rapport with the candidate. This was another missed opportunity.

EACH ROUND HAS A PURPOSE. Each round was intended to give the hiring manager an opportunity for deeper exploration and to gain additional perspectives. With each round, they were encouraged to focus on certain criteria and dig deep until they could make a determination for those criteria.

FIRST INTERVIEW. The first interview was intended to weed out candidates who lacked the necessary drive for results. Without sufficient drive, the rest of the qualifications did not matter, so it was important for the hiring manager to advance only candidates who demonstrated that they would hustle, persist, and overcome challenges. The STAR questions were ideal for this type of interview.

SECOND INTERVIEW. The second interview involved bringing in a fresh perspective from a fellow RSM. Having established that the candidate has sufficient drive, the second interview should focus more on aptitude. What better way to gauge aptitude than through a sales audition?

example interview process guidance for Interview 1 & 2

THIRD INTERVIEW. The third interview was in person, typically at a coffee shop or restaurant. This was an excellent opportunity to gauge the candidate’s presence as well as their strategic thinking, planning, and decision-making skills. The candidate could be asked to bring a 30-60-90 day plan, for example, and discuss their thought process.

Design Principles at Work

This case study illustrates the real-world application of four design principles related to skill-based hiring:

1. Target Candidate Profiles. Where there is some historical track record for a role, it should be easy to identify the candidates who later became the most successful hires, and from that, to abstract 3-8 archetypes that can be fleshed out into target candidate profiles. These profiles give recruiters and hiring managers a clear sense of WHO they are looking for, which helps them figure out where to find them and how to interact with them.
2. Clear Hiring Criteria. While a job description, a skills map, or a competency model may contain a plethora of attributes needed to succeed in a particular role, it’s important that we narrow down the criteria that best predict which candidates will become top performers. This gives recruiters and hiring managers clear guidance regarding WHAT they should be looking for during the interview process. For any role, this usually comes down to 3-5 hiring criteria that are strong predictors of success.
3. Stage-Gated Interview Process. There is a tendency for an interview process to become a long and arduous check-the-box exercise, where hiring managers feel like they are going through the motions, and candidates feel like they are answering the same questions again and again. Instead, each round of interviewing should be designed with a clear purpose in mind, with specific goals and criteria that the interviewers are trying to accomplish on their way to making a final decision. Each round should feel different, dig deeper, and add value; otherwise, eliminate the extra work.
4. Skill-Based QuestionsUsing STAR questions, in which candidates reflect on a specific situation and describe their responsibilities, thought process, actions, and results, is a far more effective way to assess a candidate’s capabilities than traditional interview questions that can easily be found online (e.g., “Tell me a weakness.”). An even better approach combines structured questions with job-relevant auditions, allowing candidates to prove their capabilities in action rather than simply talk about them.

We welcome your perspective on other ways to make job interviews more effective and impactful. Drop us a line!

Videos