AI-Driven Pre-Hire Assessments: A Buyer’s Guide

Shiran Danoch | I/O Psychologist

September 10, 2020

Executive Summary

As HR and talent leaders consider how to improve recruiting efficiency, assessments can be a strong way for talent teams to leverage technology to drive change. The following guide explains why assessments are an important part of a streamlined, data-driven hiring process and how to evaluate various assessments and determine the right fit for an organization.

Introduction: Why add an Assessment to your hiring process?

As companies increasingly rely on technology to support, enhance, or drive their business, HR and talent leaders are looking for ways to leverage technology to improve processes. Offering online applications is the new standard, making it easier than ever for candidates to submit their CV for a role. According to Glassdoor, corporate enterprises, on average, will receive 250 resumes per open role. With record number of CV submissions per year, it’s important to manage limited internal resources and optimize the screening process. On top of this, competitive job markets make quick decisions is critical.

Saving time and increasing efficiency

Recruiters’ time is very precious, and CV sifting, review, and phone screening is very time consuming. Many companies today use algorithms to automatically screen CVs and rank candidates, often by prioritizing certain keywords. Such systems may be problematic for a number of reasons:

  1. These algorithms are often biased, as they are trained on recruiter decisions. Amazon, for example, famously deployed an algorithm whose selection bias was notoriously skewed against female candidates.These algorithms become only as accurate and biased as the people who made decisions in the past.
  2. It’s easy for candidates to “cheat” these systems simply by filling their CV with keywords featured in the job description.
  3. These systems require copious amounts of clean and mature data to ensure their accuracy.
No alt text provided for this image
Value of assessments for recruiting teams

Assessments can empower recruiting and talent teams with insights and data about candidates unavailable through CVs or phone screens. Assessments can give a clear, concise recommendation without requiring any effort from a recruiter, and they enable easy and fair comparison of candidates because measurement is numerical and standardized. Assessments also allow recruiting teams to quickly know whether a candidate meets the basic job requirements and can help prioritize candidates according to best fit.

No alt text provided for this image
Assessing Assessments: How to determine which solution is right for your organization?

As you explore various types of assessments, it’s important to consider the following elements of an assessment to understand exactly how an assessment can impact your business:

No alt text provided for this image
Research and science- driven: Does the assessment use proven science to screen and match candidates?

It’s absolutely critical to deploy a science-driven assessment. A large segment of American corporations still rely on Myers-Briggs personality testing, whose scientific validity has been seriously questioned by the academic community. How can non-academics evaluate the scientific strength of a particular assessment? It’s important to make sure that an assessment’s design is based on existing research and theories that have been validated, like The Big 5, Hogan’s personality inventory, etc., and more specifically, in the context of work and employment (not, for example exclusively in a lab or on a population or setting unrelated to work).


Validation: Are the Assessment scores validated against actual employee performance?

When evaluating assessments, it’s important to stay focused on the main objective you’re trying to achieve by bringing in an assessment in the first place: to hire a larger percentage of candidates who will be successful, will stay at the organization for long, and will be high performers. Algorithms are computer models that make decisions and “learn” from previous decisions; their power and accuracy comes from the efficient aggregation of large amounts of data. Most assessments train algorithms on their recruiters decisions, and as a result, continue to perpetuate, and even exaggerate the hiring decision inaccuracies that recruiters make. A strong assessment will train algorithms on actual performance data so that the assessment score is indicative and predictive of actual employee performance. Combined with valuable recruiter insights about the role, past hiring experience, and organizational culture, assessment data provides additional empirical data about a candidate that can fill in the gaps.

No alt text provided for this image

"A strong assessment will train algorithms on actual performance data so that the assessment score is indicative and predictive of actual employee performance."

If this seems obvious, why don’t more assessments do this? For one, it is very technically challenging because there are no standard measurements for employee performance across all organizations. With all of this variance in the data and lack of uniformity, it is technically challenging to design models sophisticated enough to take these various parameters into account. Second, if performance reviews are scored or qualitative, there are many cases where managers do not provide variance in their performance review scores. Why? Because performance reviews are so sensitive, a manager who avoids confrontation or conflict may give all employees the same or similar scores.

Third, data a single employee can be scattered across three systems: ATS, which will hold hiring data, workforce management systems, which hold HR data like compensation, retention, and manager reviews, and other systems (such as Salesforce or appraisal software that captures professional performance). This makes is incredibly difficult to actually measure whether performance prediction actually yielded long term performance results.

No alt text provided for this image
Bias-free: Does using the assessment lead to increased diversity?

Most assessments on the market today claim to be bias-free and to promote diversity. However, to evaluate whether an algorithm actually stands true to this claim, it’s critical to evaluate this in the context of how algorithms work. Algorithms make decisions and “learn” from previous decisions; their power and accuracy comes from the efficient aggregation of large amounts of data. As explained in the previous section, most assessments train algorithms on their recruiters decisions, and as a result, continue to perpetuate, and even exaggerate the mistakes that recruiters make. Algorithms trained on actual employee performance (i.e: retention, sales, customer satisfaction, quotas etc.) ensure that you are looking at “neutral” personality traits, skills, and factors to define and profile a successful employee and not allowing biased factors (ethnicity, age, gender, education, assumed socio-economic status etc.) It’s critical to ask assessment providers to share data demonstrating that the assessment does not discriminate between such groups in the candidate population.

No alt text provided for this image
Unique to my organization: is the Assessment customized or off the shelf?

Behavioral science research has advanced rapidly in the past few years, and increasingly, researchers are exploring how our understandings of human behavior can be applied to the workforce. Specifically, modern assessments now allow for the collection of data about a specific individual’s cognitive, motivational, decision-making, technical, and social skills. More importantly, it’s critical to understand how they interact, for instance, how do these various elements of an individual interact with the job requirements and environment? The power to make predictions comes from a clear understanding of a person’s specific strengths and weaknesses and matching them to the job. This means that organizations need tailored assessments that can accurately capture an individual’s strengths and weaknesses to predict whether there is strong job fit.

It’s also important to be clear which traits, skills, and competencies are being measured by the assessment and whether they’re actually relevant for the specific role. There are so many soft-skills and professional skills, and various combinations of the two, that are baseline requirements for roles. A strong assessment will not only be able to measure all of these skills but enable you to understand which combinations of these skills are the most predictive of actual on-the-job performance.

Talent leaders should consider whether the assessment is tailored enough to focus on the skills relevant for the role. Oftentimes, one-size fits all models forces a theoretical model on a population, which will yield imprecise results.

No alt text provided for this image

No alt text provided for this image
Candidate Experience: how engaging is the assessment?

Unemployment rates are an all time low, making talent acquisition all the more challenging. Organizations need assessments to be engaging and reflective of the company culture so candidates do not “drop off” in the first stages of the hiring process. Using an assessment that is automatically sent to a candidate via SMS, for example, can give employers a competitive advantage. They are reaching out to the candidates almost instantaneously before the candidate has a chance to engage with another employer. At the same time, it’s important not to sacrifice quality of assessment for the sake of engagement. The gamification or commodification of assessments, for example, can be appealing to candidates, but can sometimes lack scientific research that proves their effectiveness. The real risk is that it will take a long time to prove the validity of this method because it will require a tremendous amount of data about candidates and their long term performance.

No alt text provided for this image
Connected Data: Easily connect and unify all organizational systems

Large organizations have multiple large systems in place which makes it hard to analyze data. Additionally, hiring processes can vary by business unit. An Assessment that claims to be predictive in any way must be able to connect hiring data captured by the ATS and workforce management data. The strength of an Assessment is its ability to show precisely how hiring decisions yield impressive long-term employee performance results. The Assessment can serve as the “bridge” between multiple systems. Also, a strong assessment will enable you to easily save and access employee data in an organized and structured fashion. Recruiters have tremendous insights about candidates, but if the input is qualitative in free form text, there’s no way to search or analyze in the future. Structured data is critical to conducting analysis and deriving insights.

No alt text provided for this image
Business Impact: Clearly demonstrating ROI in dollar value

HR and talent teams increasingly want to be seen as valued partners to the business. As this function continues to transition and evolve from “administrator” to “advisor,” it’s incumbent on HR and talent leaders to be thoughtful about how to best communicate the impact their initiatives have on the business overall. One of the clearest ways to do this is to “speak” the language of the business and “translate” activities into ROI.

No alt text provided for this image

A strong assessment should help an organization identify and hire better, higher-performing employees, and as a result, the business should see an improvement in business results (revenue, customer satisfaction, R&D innovation etc.) This impact must be quantifiable and presentable to business leaders so that they can clearly understand how hiring choices drive impact. A strong assessment provider will also usually provide extensive analysis and communication tools (dashboard, reports, business impact summaries etc.) that can be circulated to relevant stakeholders.

No alt text provided for this image
Privacy Compliance: GDPR and Privacy Law

This is a baseline requirement for all assessments- considering that assessments collect personal data, it’s imperative to be in line with legislative requirements. Given bureaucratic overhead, it’s not worth considering a tool that is not compliant from the get-go.

No alt text provided for this image
Conclusion

Assessments can be a valuable tool to maximize limited recruiting resources, improve candidate experience, and clearly demonstrate talent initiative impact on company overall-performance. Deploying this technology can also unify HR systems and empower talent teams with new data and insights, further positioning them as trusted leaders and advisors in the organization.