What Is Incremental Validity? Hiring Examples Explained
By Synopsix · November 11, 2026 · 13 min read
What if a test that predicts job performance well is still a waste of money?
Incremental validity is the extra accuracy a new predictor adds beyond the predictors already in use. You measure it as the rise in explained variance when that predictor joins the existing model. The same pattern shows up in offender-risk research. When researchers compared several instruments for serious violent offenders, only one still added predictive value once the established tool was already counted.
Hiring leaders often ask whether an assessment is good, then treat its standalone accuracy as the answer. A score can track job performance closely and still be worth nothing to your process, the way a second thermometer on the same wall only repeats the first reading. Your current tools already capture what the new score measures, so you pay for a sharper brochure and get the same decision.
The figure that decides whether a purchase earns its place is the size of the slice the tool adds. That slice is often thinner than the sales material suggests. Check the gap first, before you buy a new assessment or renew an old one.
What Incremental Validity Means in Plain Terms
Validity is how well a score predicts the outcome it's meant to predict: job performance, first-year grades, or whether a new hire is still with you in year two. When a cognitive test correlates with performance ratings, that correlation is its validity. It tells you the scores carry real information about the future.
Incremental validity asks a tighter question. Does this new score predict something your current measures miss? It's the difference between a test that works and a test that works for you.
Four terms carry the whole idea.
Say a hiring panel already runs a cognitive test and a structured interview. A vendor pitches a personality questionnaire with solid published research behind it. The panel's real question isn't whether the questionnaire predicts performance in general. It's whether it predicts performance after the cognitive test and the interview have done their work. If the questionnaire mostly measures conscientiousness, and the interview already surfaces conscientious behavior, the gain will be thin.
> The one-sentence test: a tool is being judged on its own strength when the pitch cites its validity, and on what it adds when the pitch cites its incremental validity against your baseline.
By the end of this guide you'll be able to say which of those two questions a supplier is answering, and whether it's the right one for your team.
Why a Second Test Only Helps If It Sees Something New
A clinic already takes every patient's blood pressure. Someone proposes a second screening test. The budget committee's question isn't whether the second test is accurate. It's whether the second test flags patients the blood pressure reading misses. If both tools catch the same people, the clinic pays twice for one insight.
Hiring works the same way. A structured interview and a cognitive test each predict job performance reasonably well alone. If they mostly capture the same underlying ability, the interview adds a little, and sometimes nothing measurable. You're running two tools to learn one thing.
Overlap and the unique part
Every new predictor's contribution splits into two pieces. The overlap is the share of the outcome that both the new predictor and the existing ones already explain, and at the margin it's dead weight. The unique part is what only the new predictor sees. Incremental validity is that unique part, measured against the variance still left unexplained.
A concrete version. Two candidates score identically on the test. In the interview, one describes how she defused a dispute between two teammates, and the interviewer rates her conflict handling highly. That rating carries information the test couldn't produce, and conflict handling predicts how a team performs six months in. That's where the gain lives, in information that isn't redundant.
The same logic shows up far outside the workplace. A 2025 longitudinal study in the [Journal of Child Psychology and Psychiatry](https://onlinelibrary.wiley.com/doi/full/10.1111/jcpp.14181) followed 1,216 justice-involved adolescents for seven years and found that callous-unemotional traits incrementally predicted several early-adulthood outcomes beyond antisocial-behavior trajectories. Two measures that sound related explained different parts of the future.
Here's how that plays out in a selection process.
<iframe width="100%" style="aspect-ratio: 16 / 9;" src="https://www.youtube.com/embed/ygZhN-2Cpqs" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen></iframe>
Three conditions raise incremental value, and all three have to hold.
When you [build a hiring assessment test](https://synopsix.ai/blog/hiring-assessment-test), the design question is which signals are non-redundant, not which single instrument has the highest coefficient. Look for information your current tools can't see. Strong numbers alone tell you nothing about what's left to discover.
How to Calculate and Read the Incremental Gain
Five steps, in order. Run them on your own data with your own criterion, not on a vendor's sample.
1. Fit the baseline model. Use only the predictors you already run. Regress the outcome on them, record the multiple correlation R, and square it to get R-squared. That figure is what your current process explains. 2. Add the new predictor. Refit the same model with the new score included and everything else held constant, then record the new R-squared. 3. Subtract. Change in R-squared equals the new R-squared minus the baseline R-squared. That difference is the incremental gain. 4. Test the added term. An F-test on the new predictor tells you whether a gain that size could plausibly be zero in the population. Where you have the headcount, confirm it on a holdout sample you didn't use to build the model. 5. Judge the size. Significance only says the gain isn't noise. It says nothing about whether the gain justifies a second assessment, which is the next question.
Now a worked example with illustrative figures. Suppose a cognitive ability test alone correlates 0.51 with first-year performance, so R-squared is 0.26. A structured interview alone correlates 0.38, so R-squared is 0.14. Put the two together and the multiple R is 0.58, R-squared 0.34. The interview adds roughly 0.08 on top of the cognitive baseline.
| Predictor set | R-squared | Gain over cognitive baseline | | --- | --- | --- | | Cognitive ability test alone | 0.26 | Baseline | | Structured interview alone | 0.14 | Not applicable (not the baseline) | | Cognitive test plus structured interview | 0.34 | +0.08 |
Read that table twice, because it hides a trap. The interview looks stronger than the increment suggests, and the cognitive test looks weaker than its 0.51 correlation implies. Run the model the other way round, with the interview as the baseline, and the same pair of tools produces a different gain number. The combined R is fixed. The gain depends entirely on what was already in the model.
> Order matters more than strength: a predictor's own validity tells you nothing about its increment. Only the baseline tells you that.
One more caution before you trust a number. If the scores feeding the model are unreliable, the arithmetic will flatter the new tool. Check [assessment validity and reliability](https://synopsix.ai/blog/assessment-validity-and-reliability) first, then run the five steps on your own hiring data before adding anything new.
How Large a Gain Counts as Meaningful
A raw change in R-squared is hard to read alone, because its size depends on how much the baseline already explains. The effect size f-squared corrects for that by dividing the gain by the variance still unexplained: f-squared equals the change in R-squared divided by one minus the full model's R-squared.
Apply it to the worked example. The gain is 0.08 and the full model's R-squared is 0.34, so the denominator is 0.66. That gives an f-squared of about 0.12, which sits in the small band of Cohen's conventions. Small is the right label. Anyone presenting that as a strong improvement is rounding in their own favour.
| Effect size (f-squared) | Label | Practical reading | | --- | --- | --- | | Below 0.02 | Negligible | Do not add the predictor on incremental grounds alone | | 0.02 to 0.14 | Small | Keep only if the time and cost of the tool are low | | 0.15 to 0.34 | Medium | Defensible when the criterion is expensive to get wrong | | 0.35 and above | Large | A clear addition to the process |
Significance and practical size answer different questions, and teams blur them constantly. With a very large sample, a tiny gain can clear the F-test and still not justify a second assessment. With a small sample, a meaningful gain can fail the test because you lacked the numbers to detect it. Neither result tells you what to do on its own.
> Set the bar before you look: decide the smallest f-squared you'd act on while the study is still being designed. Deciding afterwards invites you to rationalise whatever number arrives.
Then consider scale. A gain that looks tiny on paper can still matter to a company hiring hundreds of people a year, because small improvements accumulate across every decision the model touches. Ten extra correct calls out of a thousand is a rounding error to one manager and a serious saving to a recruiter working at volume.
Weigh the gain against what the tool costs in hours, licence fees and candidate experience. Then keep it, or don't.
Real Cases from Hiring and Offender Risk Research
A graduate scheme already screens on academic grades and a cognitive test. A vendor then offers a personality measure with decent published validity. Run one check before anyone buys it: does the measure predict first-year performance once grades and test scores are already in the model?
If that extra prediction lands near zero, the measure is valid and redundant at the same time. The instrument is not broken. It works like a second thermometer in the same room: it reports a temperature you already have. A strong standalone coefficient can still add nothing once the tools you already run cover the same ground. Checking that gap is the first step before you buy a new assessment or keep one that is already in the process.
Risk assessment follows the same pattern, and a wrong call costs far more. In a 2025 study of 420 Australian male serious violent offenders, several instruments were compared and only one added incremental predictive value beyond an established tool, as recorded in the [APA PsycNet entry for the study](https://psycnet.apa.org/record/2025-92149-001). Several credible tools. One that saw something the established tool had not already covered.
Why a sales role can behave differently
A company hiring account executives often has no past sales results for external candidates. The baseline is thin: a CV, a conversation, maybe a reference. A role-fit assessment can add real gain there because it supplies information the team lacked. The same kind of tool would look redundant on the graduate scheme, where grades and a cognitive test already carry much of the signal.
Set the two cases side by side and the rule does not change with the stakes. Ask what this tool knows that you do not already know.
> Expect nulls. In a mature process with good coverage, most new tools will add little, and that's a statement about how well your current process works, not a verdict on the instrument.
Treat a null result as a useful answer. You learned that your existing tools already carry that signal, and you learned it before you spent budget and candidate goodwill on a second round of testing.
Confirm Added Value Before You Adopt a Tool
Put five questions to any provider before you sign or renew.
1. What is the gain against my baseline? Ask for incremental validity evidence measured against the predictors you already run, not a standalone validity coefficient with an impressive number attached. 2. Which outcome did you predict? If the study predicted supervisor ratings but you decide on retention, the gain may not transfer to your decision. 3. Was there a holdout sample, and how large was it? A model tested on the same data that built it will always look better than it behaves next quarter. 4. Can you show what drives an individual result? A score nobody can explain won't survive a hiring manager's challenge. Explanations of the key factors behind a user's results are available from Synopsix on request. 5. What does it cost in time and candidate experience? A long test that pushes strong candidates to withdraw can erase a small gain. Run that against a full [cost benefit analysis for AI projects](https://makeautomation.co/cost-benefit-analysis-framework/) rather than counting the licence fee alone.
Headline accuracy figures are claims until your team tests them on your own hires. That applies to new purchases and to the [validated assessment tools](https://synopsix.ai/blog/validated-assessment-tools) already in your stack, whose increment shifts whenever your process changes around them.
The decision rule is short. Keep a tool only when your team can show a meaningful, tested gain over what it already uses.
---
Synopsix reads each candidate against the role, the team and the culture rather than a generic profile, which is exactly the kind of signal worth putting through the increment check. Bring your current predictors, run the comparison on your own hires, and keep only what earns its place. Visit [Synopsix](https://synopsix.ai) to see the profiles and reports your team would work from.