Creating AI-driven pre-employment assessments

What are AI-based assessments?

HireVue AI-based Assessments are pre-employment assessments designed by people for people. The purpose of Assessments is to give recruiters and TA teams a standard, structured, and fair way to quickly screen more candidates.

• Assessments don’t replace recruiters. They simply help recruiters and talent acquisition teams assess more candidates more quickly and more accurately.
• Assessments are like putting more reviewers to work for you without adding more people to your team.
• HireVue democratizes the hiring process by minimizing bias in our algorithms.

What AI-based assessments measure

HireVue employs a team of industrial-organizational (IO) psychologists and data scientists. This team develops assessments that target the skills needed for a variety of roles. That gives TA teams assurance that candidates have the requisite competencies needed to perform the job, are likely to be higher performers, and are potentially more likely to stay with the company.

Assessing for ability to work with others, ability to do the job, and work style and personality gives recruiters and TA teams a 360-degree view of a candidate.

The human side of developing AI-based pre-employment assessments

But how does AI measure these competencies? Loren offers another nice summary of the steps that go into developing Assessments in, “Train, validate, re-train: How we build HireVue assessment models.”

What Loren doesn’t cover, is that when HireVue develops algorithms that score our assessments, we work with real people. We ensure the algorithms reflect how expert human raters assess job-related competencies.

We ensure that there’s a strong correlation between the human raters’ assessments and the algorithm.

We don’t stop with assuring the algorithm matches the assessment of human raters, though. We go on to ensure those assessments actually predict job success. We also follow the Equal Employment Opportunity Commission (EEOC) Uniform Guidelines when developing, testing and monitoring all HireVue AI-driven assessments. The Guidelines require fair treatment of applicants in the hiring process regardless of race, gender, and age (40 and over). These protections include the 4/5ths Rule, also known as the Red-Flag Rule and other statistical tests for group differences; all of which HireVue uses to monitor and mitigate our assessments.

If we see that one group passes the assessment at a significantly different rate compared to another group, known as adverse impact or algorithmic bias, we drill in and determine why. We then remove or minimize any data points that lead to that bias to ensure proportional outcomes for all groups and protected classes.

For instance, if we see that a disproportionate number of men score higher than women, we determine what’s causing the bias and mitigate for it. We repeat the mitigation process until men and women perform similarly enough to confirm that the bias is minimized.

In addition to EEOC guidelines, at HireVue, we test for a wider array of group differences, beyond those required by law. That’s because there are biases beyond age, gender, and ethnicity, for example attractiveness, country of residence, non-native accent, etc.

We release an algorithm for customer use, only after we’re confident that race, gender, and age group differences (or adverse impact) have been minimized. And even after it’s released, we continue to monitor the algorithm for any bias creep.

The goal of AI-based assessments

The goal of AI-based assessments is to:

For example, is the candidate who applied for a customer service role good at working with people? Or is a candidate who applied for an accounting role good at working with numbers?

Candidates aren’t assessed for a single competency. They’re assessed for multiple competencies, cognitive abilities, and personality traits, all of which are shown to contribute to success on the job. Their scores are shared with the recruiter and the candidate. That feedback can be used in hiring decisions. It can also be used post-hire as a development opportunity.

Seeing the whole candidate

HireVue offers a broad suite of assessments, including video interviewing, game-based assessments, and coding challenges. These can be combined in a modular fashion for specific roles. Using more than one type of assessment offers divergent validity, meaning each measures for different things. For example, you can combine an assessment that assesses for soft skills with one that assesses for hard skills.

HireVue’s library of competencies can be mixed and matched to assess the general competencies needed for the role you’re filling.

For example, a HireVue assessment for call-center representatives will, at a minimum, assess for communication skills, ability to learn, and work ethics which are the key competencies needed to be successful in a call center role. We might also include questions to measure dependability, problem-solving, and negotiation/persuasion given the specific requirements of the call center role for a given customer (for example, a Sales-focused representative). The assessment would include 3 to 4 AI-based interview questions and 2 to 3 game-based questions for a total of 5 to 7 questions in the assessment (15 to 25 minutes completion time).

A report of results shows recruiters the candidates’ overall assessment score and their specific competency scores. Also, recruiters see all candidates’ scores stack-ranked against each other. Candidates also have the ability to get a report of their assessment results.

AI-driven assessments work and can be trusted

AI actually powers a more accurate, less biased, more engaging screening process for both recruiters and candidates.

The HireVue team strives to eliminate as much bias as possible. It strives to deliver on the goal of reducing time to hire and increasing performance and retention. In the session, we shared some case studies that show that Assessments work. Estee Lauder, for instance, documented an increase in the average sales made for sales clerks. That increase directly correlates to the clerks’ scores on their HireVue Assessments.

Hear more on the topic in our session from Horizon: HireVue’s AI-Driven Assessments: How Do They Work and Should I Trust Them?

About the authors

Dr. Nathan Mondragon is Chief IO Psychologist at HireVue where he builds, researches, and maintains the AI-driven HireVue Assessment product. Nathan has more than 20 years of experience in the human capital management (HCM) space and is a recognized expert at blending IO psychology tools and technology to deliver seamless, integrated recruitment, hiring, and development solutions. In 1996, Nathan helped lead the creation and delivery of the first-ever online selection assessment. In 2004, he built, from the ground up, the first integrated assessment solution within an enterprisewide applicant tracking system (ATS) (Taleo). And in 2015, as part of HireVue, he delivered the first-ever AI-driven pre-hire assessment solution. Nathan received his Ph.D. in industrial-organizational psychology from Colorado State University. Find Nathan on LinkedIn.

John Slifka is an IO Psychology Consultant at HireVue. He works with clients to design, validate, and implement cutting-edge AI-driven assessment solutions. He brings a wealth of knowledge and expertise in HR consulting to HireVue. Prior to joining HireVue, John worked at IBM, Anheuser-Busch, and Aon. John has a passion for delivering pre-hire assessment solutions and utilizes his previous experiences and education in client-centered consulting. John earned his MA in industrial and organizational psychology from the University of Missouri, St. Louis. Find John on LinkedIn.