How AI Assessment Tools Affect Job Candidates’ Behavior

Summary:

When considering AI assessment tools, most companies focus on the gains the tools bring in terms of efficiency and quality. But what they don’t factor in is how AI assessment may change candidates’ behavior during the process.

According to the World Economic Forum, more than 90% of employers use automated systems to filter or rank job applications, and 88% of companies already employ some form of AI for initial candidate screening. Take Unilever, for example. The consumer goods giant uses AI-driven tools from HireVue to assess early-career applicants, saving 50,000 hours and more than $1 million in the process.

Most companies, when considering AI assessment tools, focus on the gains the tools bring in terms of efficiency and quality. But what they don’t factor in is how AI assessment may change candidates’ behavior during the assessment. Our new research, examining over 13,000 participants across 12 studies, reveals that this is a crucial blind spot. We looked at simulation of a variety of assessment situations in both the laboratory and the field, and we collaborated with a startup platform offering game-based hiring solutions called Equalture.

The results show that job candidates consistently emphasized analytical traits when they believed AI was evaluating them, while downplaying the very human qualities—empathy, creativity, intuition—that often distinguish outstanding employees from merely competent ones. This drove candidates to present a different and potentially more homogeneous version of themselves, in turn affecting who was likely to succeed in an AI-enabled hiring process, with implications for organizations using AI in hiring, promotions, or admission decisions.

Why This Matters for Your Organization

The implications of our findings extend beyond individual hiring decisions. When candidates systematically misrepresent themselves, organizations face several critical challenges:

Talent pool distortion: While AI is sometimes blamed for making biased hiring decisions (for example, discriminating against women in the selection process), our research suggests that knowing that one is assessed by AI also biases candidates, making them believe that they should prioritize their analytical capabilities. As a result, companies may be screening out exactly the candidates they need simply by using AI: that innovative thinker or emotionally intelligent leader you’re looking for might present themselves as a rule-following analyst because they believe that is what the AI wants to see.
Validity compromise: Assessment tools are only as good as the data they collect. When candidates strategically alter their responses, the fundamental validity of the assessment process might be undermined. Organizations may no longer measure authentic capabilities—instead, they may measure what candidates think AI will value the most.
Unintended homogenization: If most candidates believe AI favors analytical traits, the talent pipeline may become increasingly uniform, potentially undermining diversity initiatives and limiting the range of perspectives in organizations. Companies like IBM and Hilton, which integrate AI into both hiring and internal promotion systems, must now contend with whether such tools nudge employees toward formulaic self-presentation.

New transparency regulations like the EU’s AI Act, which require organizations to disclose AI use in high-stakes decisions, make these outcomes all the more likely. When candidates are aware that an AI is assessing them, they are more likely to change their behavior.

What Leaders Can Do

Based on our findings, organizations can take several concrete steps to address the AI assessment effect:

Radical transparency: Do not just disclose AI assessment—be explicit about what it actually evaluates. Clearly communicate that your AI can and does value diverse traits, including creativity, emotional intelligence, and intuitive problem-solving. This might include providing examples of successful candidates who demonstrated strong intuitive or creative capabilities. Currently, few companies seem to be transparent about what exactly it is that AI assesses—at least this information is not easily accessible when clicking through career page information on the websites of many major companies. That said, applicants discuss and share their intuitions on blogs and videos, which may be counterproductive because it may or may not align with actual practices. We advise companies not to leave their candidates to speculate.
Regular behavioral audits: Implement systematic reviews of your AI assessment outcomes. For instance, New York City has enacted Local Law 144, requiring employers to conduct annual bias audits of AI-based hiring. In response, one of the market leaders in AI-based hiring, HireVue reports their recent audits for race or gender bias across jobs and use cases. In addition to examining biases regarding demographics, we suggest using these audits to look for patterns indicating behavioral adaptation: Are candidates’ responses becoming more homogeneous over time? Are you seeing a shift toward analytical presentations at the expense of other valuable traits?
Hybrid assessment: Some organizations combine human and AI assessments. For example, Salesforce notes that besides technology, a human will review applications. Nvidia and Philip Morris International guarantee ultimate assessment and decision-making through a human. One of our studies shows that while this hybrid human assessment does reduce candidates’ tendency to highlight analytical capabilities, it does not eliminate it. To close the gap, you need to train your human hirers to compensate for the AI effect.

The Path Forward

As AI becomes increasingly embedded in organizational decision-making, we must recognize that these tools do not just change processes—they change people. The efficiency gains from AI assessment may come at the cost of authentic candidate presentation and, ultimately, the human diversity that makes organizations innovative and resilient. The irony is striking: In our quest to remove human bias from hiring, we may have created a system where AI introduces a new form of bias. The solution is not to abandon AI, but to design assessment systems that account for and counteract these behavioral shifts. Only by keeping humans—not just metrics—at the heart of our assessment strategies can we build hiring systems that truly identify and nurture the diverse talent our organizations need.

Explore AAPL Membership benefits.