| GUEST PROFILE Mark Smith, Ph.D., Principal Consultant, A Better Choice Hire. Ph.D. in Industrial-Organizational Psychology. Connect with Mark Smith on LinkedIn |
This episode was hosted by Manjeet Kaur, Xobin.
Table of Contents
TL;DR – Key Takeaways!
- Most hiring criteria are assumptions, not evidence. A degree shows someone finished something hard, not that they can do this job.
- Cognitive ability and skills are not the same. Ability predicts long-term potential; skills show what someone can do right now.
- Bad hires rarely lack knowledge. They lack teamwork, safety discipline, or cultural fit, traits a resume can’t catch but a psychometric assessment can.
- Validation helps confirm what’s true instead of leaving you to guess. Criterion-related validation proves an assessment actually predicts job performance.
- Skills-based hiring is widely declared but rarely practiced. Most employers say they’ve adopted it, yet the data shows barely any non-degree hires are actually happening (HBS/Burning Glass Institute).
Mark Smith spent 25 years helping organizations build and validate assessments before most companies were thinking seriously about either. His PhD is in industrial-organizational psychology, and his career has run through HR consulting, SHRM and, most recently, a direct invitation to testify before the US House of Representatives on skills-based hiring.
In Xobin Talks Episode #20, host Manjeet Kaur explores the real-world side of hiring assessments. The conversation looks at what these tests truly measure, why proper validation matters, and where skills-based hiring still misses the mark. They also explain how AI is reshaping the way companies assess candidates and make hiring decisions.
“What Do 25 Years of Assessment Experience Reveal About Hiring That Most Managers Miss?”
Mark opens with the observation that defines the whole field: when he started, most assessment work focused on cognitive ability. The idea that you could assess personality, safety orientation, or other non-cognitive traits in a hiring context was actively contested.
Mark: “When I started out, most assessments focused on how well someone could think, what they knew, and how well they could apply their skills. As I started to get into it more, the assessments have also developed on the other side: the personality, the softer competencies that aren’t cognitive in nature but are very important. Safety orientation and how well you team with other people, those are highly relevant, just as relevant as the cognitive stuff.”
Mark’s 25 years in the field have shown him repeatedly that the research is consistent: combining cognitive and non-cognitive assessments outperforms either approach alone. That is why his own work has always covered both sides. Among its most significant findings: structured interviews may be the strongest single predictor of job performance, and assessments that combine cognitive ability with non-cognitive measures consistently outperform either approach alone.
A resume gives you a snapshot of a candidate’s past, but it does not truly assess their skills. Candidates describe their own experience, so the document shows what they have done rather than what they can actually do or how they may perform at work. Most hiring managers know this, but many still struggle to turn that knowledge into a clear process that leads to better hiring decisions.
“How Is a Skill Assessment Different From a Cognitive Ability Assessment?”
This is the distinction Mark comes back to throughout the conversation because it shapes everything else: what you should assess, when to assess it, and what you can actually do with the results.
Mark: “Cognitive ability, that’s something else because that’s hard to develop. If a company is focused on cognitive ability you really have to assess that in your applicants and bring in the right people. If you’re talking about skills and knowledge, those are the things you can go through training, get experience, and really increase your competency in.”
Two Questions, Not One
The difference has direct consequences for how organizations should use each type of assessment. Cognitive ability is relatively stable across a career. Assessing it tells you something about long-term potential, the ceiling of complexity a person can handle, and how quickly they will learn new things. You assess it in hiring because it is difficult to change after someone joins.
Skills are different in an important way. They are trainable, improvable, and context-specific, and assessing them tells you what someone can do right now. Assessing existing employees helps you identify where training can make the biggest difference.
Mark: “Want to find out how strong your team is at a specific skill? Give them a quick assessment. The results can show you where the team needs improvement and which employees may benefit from additional training.”
The implication for organizations is that cognitive ability and skills assessment answer different questions and should be used at different points in the talent lifecycle. Treating them as interchangeable, or using only one, leaves information on the table.
“Why Does a Great Resume Not Always Mean a Great Hire?”
This is the question Manjeet raises from the candidate screening side, and Mark’s answer reframes what hiring is actually for.
Mark: “When managers review a poor hire, they often realize that the person’s skills weren’t the real issue. The employee might have known the work well but still struggled on the job. They could have had trouble working with coworkers, ignored safety procedures, or failed to meet workplace expectations. Hiring someone who knows the job is only part of the process. You also need people who can work with others, respect the rules, and add value to the team.”
Knowledge and ability predict task performance, while personality and non-cognitive traits predict whether someone actually thrives in a role. The person who gets fired almost always has sufficient knowledge. They failed on something a resume could never have predicted, and that a psychometric assessment could have surfaced before the offer went out.
The Safety Assessment Story
Mark: “A company had clients who were dealing with dangerous jobs where safety was terribly important. The client wanted a measure of safety orientation: to hire people who weren’t Mavericks, who weren’t willing to bend the rules, who would follow safety rules and therefore be safer. What we were able to show is that the people they hired based on the safety orientation assessment later had fewer actual safety incidents.”
Hiring managers who disliked the filtering effect pressured HR to remove it, and the company complied. Safety incidents went back up when the assessment was removed and dropped again after reinstatement, a quasi-experiment Mark could never have designed, and the clearest possible evidence the assessment was measuring something real.
“What Is Validation, and Why Should a Hiring Manager Care About It?”
Most people in HR have heard the word validation used in a dozen different ways. Mark cuts to what it actually means from an industrial-organizational psychology perspective.
Mark: “Validation shows that an assessment measures skills that matter for the job. Criterion-related validation: it’s relatively straightforward but it’s a tricky process. You get a group of job incumbents to take an assessment, you get a measure of job performance for the same people, and then you can statistically correlate: does how well somebody performs on this test predict who is a good performer?”
Criterion-related validation connects assessment scores to real outcomes: not opinion, but data showing that higher scorers actually performed better at the job.
The practical implication is accountability: an unvalidated assessment in a hiring battery is a guess. Without validation data showing that an assessment predicts performance in your specific context, there is no defensible reason to use it and, in some jurisdictions, real legal exposure if you do.
Mark: “The engineers would always approve the process. But I’ve also seen engineers try to run the process themselves and they just don’t know what reasonable expectations are for what sort of magnitude of relationship you should be able to find. The engineers like to talk about the process but they need some help actually running a validation study.”
This is not a knock on engineers. It is the honest observation of someone who has run hundreds of these studies: the process requires knowledge that is domain-specific and easy to get wrong in ways that are invisible until they matter.
“Why Did the US Congress Ask You to Speak About Skills-Based Hiring?”
Skills-based hiring has moved well beyond HR conference discussions and is now a legislative priority.
Mark: “Congress in the US is made up of two chambers: the House and the Senate. One of the committees on the House side is the committee on Education and the Workforce. One of their priorities has been what they’re referring to as skills-based hiring. They decided to have a hearing and they were looking for experts to talk to, and I don’t know how they got my name but they got my name and I said I would love to testify.”
The Core Problem: A Degree Is Not a Measurement
Mark: “A person earns an accounting degree from a university. The assumption is they have a high level of knowledge in basic accounting. But some people graduate just barely, some graduate with a lot. So the assumption that just because you graduated doesn’t necessarily mean you have the skills. We can directly assess those skills.”
Most employers say they use skills-based hiring, yet Harvard Business School and Burning Glass Institute found that at some large firms, fewer than 1 in 700 hires were non-degree graduates even after requirements were dropped (HBS/Burning Glass Institute). The policy changed, but hiring teams still used the same approach. They now need a better way to assess candidates than relying on degrees.
Two Paths Forward
One is employer-run assessments: companies design or license tests that directly measure skills relevant to specific roles. The other is certification programmes, where individuals earn verified evidence of competence from non-university providers.
Mark: “When hiring for IT roles, companies often look for relevant credentials and certifications. Microsoft has a bunch of tests for IT people to certify on different products at different levels. Google does the same thing. That takes the onus away from the hiring organization and moves it to the individual.”
Both paths are valid and both require the same thing: direct, validated measurement rather than inference from a degree.
“How Can AI Improve Assessments, and What Risks Should You Keep in Mind?”
Mark ends by raising the question that has become a major focus for the assessment industry since the episode was recorded.
Mark: “The future will talk a lot about AI and how this works with assessments. It’s tricky, it’s not necessarily straightforward and we’re going to run into a lot of laws in different states and countries. Europe is going to have different sets of rules about the use of AI when it comes to assessments. The next phase is about helping companies handle these changes and use AI the right way throughout the process.”
The practical concern is straightforward: if candidates can use AI to complete assessments, those assessments no longer measure what they are designed to measure. A 2026 study in the International Journal of Selection and Assessment found that fewer than 3% of applicants self-reported using generative AI on pre-hire assessments in 2024. When AI combined with other algorithmic tools was included, that figure rose to 19% (Robie et al., 2026). Warning statements reduced reported use, but the problem is not going away.
The design response is to develop assessments that are harder to game with AI: oral components, live demonstrations, situational judgment items that require contextual reasoning, and proctored environments that make AI assistance impractical. Mark’s broader point: AI in assessments is not inherently bad, but it needs to meet the same standard as everything else in the field, does it improve or undermine the validity of what you are measuring?
🎧 Watch the Full Episode
Xobin Talks – Episode 20 | Mark Smith, Ph.D., Principal Consultant, A Better Choice Hire | Hosted by Manjeet Kaur, Xobin
▶ Play Episode #20 of Xobin Talks
About Mark Smith, Ph.D.
Mark Smith holds a Ph.D. in Industrial-Organizational Psychology and has spent more than 25 years in HR research, test development, validation, and leadership assessment. He joined SHRM in 2018 after the majority of his career in HR consulting, where he helped organizations across industries validate assessments and build selection systems grounded in evidence.
In 2024 he was invited to testify before the US House of Representatives Committee on Education and the Workforce on the subject of skills-based hiring, a hearing attended by members of Congress examining whether degree requirements remain a fair or effective proxy for job readiness. He is now Principal Consultant at A Better Choice Hire and serves as Xobin’s Scientific Advisor in I-O Psychology, where he helps companies implement and validate assessments for selection and development.
You’ve come this far without relying on guesswork. Now, use assessments to make better hiring decisions. Explore more conversations from Xobin Talks.
Frequently Asked Questions
What is the difference between a skills test and an ability test?
A skills test measures what someone can do now in a specific domain. A cognitive ability test shows how well someone can learn, think through problems, and find solutions. Skills are trainable and improve with practice. Cognitive ability is relatively stable across a career. Both predict job performance but answer different questions.
Why do so many bad hires have the right credentials?
Because credentials predict academic completion, not workplace behavior. When hiring managers describe a bad hire, they almost never cite insufficient knowledge or ability. They cite poor teamwork, safety violations, or cultural mismatch, traits a resume or degree cannot surface. A psychometric assessment can identify these risks before an offer is made.
What is criterion-related validation and why does it matter?
It’s the process of correlating assessment scores with actual job performance data, converting an assessment from a guess into defensible evidence: higher scorers actually performed better at this job, and there is data to prove it. It also provides legal protection where hiring assessments are regulated.
Why doesn’t dropping degree requirements lead to different hiring decisions?
Because removing the requirement from a job posting does not change who makes the final decision or what information they use to make it. The policy changes, but the underlying decision process does not. Don’t just rewrite the job posting. Test candidates on the skills they actually need for the role, not the degree they hold.
Can AI be used in pre-employment assessments?
Yes, but with caveats. A growing regulatory landscape treats hiring AI as high-risk, requiring oversight and transparency. A more immediate concern is candidates using AI to complete assessments, which undermines their validity. Proctoring, oral components, and contextual question formats are the design response.
Is psychometric testing fair for diverse candidates?
Yes, when designed and validated properly. Psychometric assessment replaces a degree filter that disproportionately excludes non-traditional candidates who are qualified through alternative routes. Validated assessments significantly expand the accessible candidate pool compared to credential-based screening alone.
How should companies start building a skills-based hiring process?
Identify the skills, behaviors, and competencies that distinguish strong performers. Select or develop assessments that directly measure those things. Run a validation study to confirm they predict performance in your specific context. Do not simply drop degree requirements without replacing them. Hiring managers default back to credentials by habit without a validated alternative.
