Episode 287 · 26 March 2026 · 51:23

What if A.I. is Fairer Than Humans? The Truth About Bias in Recruitment, with Kate Young, Head of People Science at Sapia.ai

0:000:00

Tip: click any timestamp below to jump there and keep playing.

Chapters & timestamps

Click any timestamp to jump the player to that moment.

Intro: The 6 AM flight to Munich
intro#travel #early-morning #occupational-psychologist
Who is Kate Young?
interview#occupational-psychologist #people-science #head-of-people-science
What is job analysis (and why is it so painful)?
interview#job-analysis #process #pain-points
Is job analysis still relevant in the age of ChatGPT?
interview#job-analysis #chatgpt #ai #relevance
Kate's favourite job analysis story
interview#job-analysis #story #time-consuming
How AI condenses 8 hours into 90 minutes
interview#ai #time-saving #efficiency
From competencies to questions
interview#competencies #questions #interview
Why it's not a chatbot
interview#chatbot #ai #distinction
Everyone gets an interview
interview#interview #everyone-gets-interview #hiring
Candidate satisfaction scores
interview#candidate-experience #satisfaction #hiring
Neurodiverse candidates and fairness
interview#neurodiversity #fairness #candidates
The nerdy data bit
interview#data #analytics #nerdy
No automated pass/fail
interview#automation #pass-fail #scoring
What's stopping bias at the filter stage?
interview#bias #filtering #hiring-manager
Who is Sapia working with?
interview#enterprise #ai-hiring #clients
"You're reducing somebody to a number"
interview#reducing-to-number #criticism #ai-hiring
The cost of doing it yourself
interview#cost #diy #hiring
The one thing to take away
interview#takeaway #hiring #advice
Top 3 takeaways
interview#takeaways #hiring #cost #fairness

Show notes

What if A.I. is Fairer Than Humans? The Truth About Bias in Recruitment — with Kate Young

Guest: Kate Young — Head of People Science at Sapia.ai, occupational psychologist, and former traditional psychometrics consultant.

Most businesses are making hiring decisions based on fundamentally broken data: the CV. Kate Young is on a mission to change that.

Kate joins Al and Leanne to explore what happens when behavioural science meets artificial intelligence in recruitment. She shares the "painful" reality of traditional job analysis — including her 6:00 AM flight to Munich to shuffle cards with 30 stakeholders for eight hours — and how AI has condensed that process into 90 minutes of high-precision data.

They dig into the death of the CV, why "blind" chat interviews give every candidate an equal shot, how to strip ableist and majority-group language out of job descriptions, and why AI isn't replacing psychologists — it's amplifying them.

Top 3 Takeaways

Your hiring process is only as good as your job analysis. If you don't define what a person actually does all day — the tasks and behaviours — no hiring tool can save you. Measure what matters.
Bad hiring isn't just expensive, it's unfair. We've romanticised the human touch in recruitment, but unstructured interviews default to the "loudest voice in the room." Standardisation is the simplest way to reduce bias.
The loudest voice in the room is probably the least reliable. Whether it's a hiring manager with a strong opinion or a candidate who interviews well, the data should always be part of the conversation.

Connect With Us

Get in touch, book a call, or find Al and Leanne on LinkedIn — all in one place: truthliesandwork.com/connect

Mental health support: findahelpline.com — UK: Samaritans 116 123 · Mind 0300 123 3393 — US: 988 — Australia: Lifeline 13 11 14

Truth, Lies & Work is part of the HubSpot Podcast Network.

Full transcript

Expand full transcript

Al ElliottIt's 6 AM one cold winter morning, and an occupational psychologist is running to catch a flight to the patent office in Munich. When she lands, she spends the next 8 hours in a room with 30 patent officers shuffling cards around a table, trying to figure out what makes a great patent officer. And as you can imagine, everyone had their own opinions, and most of those were different.

Kate YoungTrying to get 20 or 30 people to coalesce around that, who didn't really understand what the qualities on the cards were in the first place, had very different opinions. And kind of to my earlier point, with hiring managers always believing they know the job had their individual biases was a difficult exercise that I'm not really sure got us to a good outcome.

Leanne ElliottThat occupational psychologist was Kate Young, who is now Head of People Science at Sapia.ai, and she's about to explain why AI can now do the same work as an 8-hour, 30-person international flight exercise in 90 minutes from a laptop.

Al ElliottBut Kate isn't here to tell you that AI is going to replace your hiring process or your hiring team. She's here to explain why most businesses are making hiring decisions on fundamentally broken data and why the fix is a lot simpler than you think.

Kate YoungYou have to measure what matters, which is the right competencies or skills for the role, whatever language we prefer. But then you have to ask the right questions that are going to feel fair and inclusive and give everybody the right opportunity to demonstrate their skills. So the AI enables this in a very clear way because it consumes the competency We've agreed on, we've chatted about, we've swapped in, swapped some out, we're all happy. And then it's going to combine that data with the job description that is really clear about what's required in the job to come up with some really tailored questions that are explicitly linked to the competencies.

Leanne ElliottHello, and welcome to Truth, Lies and Work, the award-winning podcast where behavioural science meets workplace culture, brought to you by the HubSpot Podcast Network The audio destination for business professionals. My name is Leanne. I'm a Chartered Occupational Psychologist.

Al ElliottMy name is Al, and I'm a business owner.

Leanne ElliottAnd we are here to help you simplify the science of work.

Al ElliottKate Young has spent her career in hiring science, from traditional psychometrics at some of the biggest consultancies to leading the people science team at Sapia.ai. Sapia are a pioneer in ethical AI for hiring. They're known for bringing rigorous people science and strong AI governance into recruitment at scale. They've helped brands like Costa Coffee, BT Group, and Hollander Barrett to design fair, explainable hiring systems that are grounded in structured interview science. Now, Kate made the switch into AI hiring because she saw the field moving fast and believed that without occupational psychologists in the room, it was going to go very wrong very quickly.

Leanne ElliottI have to admit, I was a little sceptical, but after this conversation, I actually think Kate's right. After this very quick break, we'll find out why your job descriptions are probably lying to you, what extroversion has to do with bias, and why the candidate who gets ghosted might actually be getting a fairer deal with AI than they've ever got before. Don't go anywhere.

Al ElliottThe dirty little secret is that most businesses think they know their customers. They have the data, the records, the history, but it's scattered across 3 teams and 4 systems. And in our case, it's about 200 116,000 spreadsheets, which means nobody can actually use it.

Leanne ElliottWhat you should be doing is using HubSpot, because why? HubSpot connects it all— every interaction, every support ticket, every conversation— into one platform every team can work from. So when sales talks to a customer, marketing already knows the full story. And when you know more, you grow more.

Al ElliottNice. Check out hubspot.com, the agentic customer platform for growing businesses.

Kate YoungI'm Kate Young. I'm an occupational psychologist by background. I'm currently Head of People Science at Sapia.ai. It's a fancy way of saying lead psychologist. Sapia are an AI hiring firm, and I've been there around a year. Previously, my career was mainly in the more kind of traditional hiring space, some of the bigger consultancies or leadership assessment firms, and more traditional hiring solutions and psychometrics. And then I made the switch over into AI hiring about a year ago because— The market and the field was moving so fast, and I firmly believed that occupational psychologists need a place in that conversation if it's to be done right, and if that industry isn't going to run away with lots of kind of technologists in their garages churning out CV screening solutions that perhaps aren't the best. So, in terms of what I'm famous for, I think the last year it's really talking a lot about that and how we make AI in hiring valid, fair, and defensible.

Kate YoungAnd I think we've already got some people kind of shifting in their seats just by mentioning AI in hiring in the in the same kind of sentence there, and we're going to get into all of it, but I want to start at the beginning and not assume that people will know what we're talking about, because some of these terms are very much in our world. So, in terms of job analysis, what is it? Why does it matter?

Kate YoungRight. So, job analysis, it's quite an old process. It's probably been around 30 or 40 years at least, if not predating the Second World War. It is a process by which you define the duties, responsibilities, tasks, behaviours, and skills that are necessary for successful performance in a job. So, a typical output might be a list of what does somebody actually do all day in this job? When they sit at their desk, what are their tasks? Or when they walk into their shop, what are their responsibilities? And then once you know what they're doing, what behaviours or qualities they need to perform that effectively? So, it might be a skill like planning. Sometimes a job analysis includes something like a value or a motivation type statement. So, this person must be highly energetic. Now, you and I as psychologists might have of feelings about that as something we should be assessing against, but sometimes that's an output. So, traditionally, a job analysis would be this quite hefty document that would be the result of many hours of focus groups, surveys, interviews with job holders, line managers, HR, to really understand what does the person do and what do they need to do it.

Kate YoungIs it something that is still relevant in a world that seems to be faster-paced, and we can ask ChatGPT to write a job description for us. Is it a dying art? Is that a problem? Where do we stand with it today?

Kate YoungI think it's even more relevant, but you're right to challenge on the pace part, because there are so many choices out there in how you hire. Everybody knows what a CV is, an interview is, and that has been around for donkey's years. And some people will know what an assessment centre is, so where you go and there's lots of people there, and they might You might be observing a group discussion, for example, or you might do role play. And most people have heard of personality tests or ability tests. So, maybe you have to answer some questions about numerical reasoning. There's lots and lots of choices in how we assess, but assessing someone fairly and correctly really comes down to measuring what matters. So, actually making sure we're measuring the right things. And the only way we can do that is a job analysis. So, with so much, honestly, bewildering choice of how you assess out there, if we're not even getting the basics right, the basics of job analysis, then we're going— we're almost sort of doubling down on that error. We're measuring the wrong things, then we might make the wrong tool choices, and then we'll get the wrong outcomes, which is potentially the wrong people for the job. And worse than that, people are unhappy in their jobs because they haven't been selected into a role that fulfils them.

Kate YoungWhat does an organisation gain by going through this detailed process to then figure out how to assess people within within that recruitment process? Because a lot of businesses will just screen resumes and maybe go to interview, maybe they'll do a work sample test. Are we just overcomplicating it as psychologists?

Kate YoungIt's a really fair challenge. And, you know, the answer is always, it depends, when you ask a psychologist any question, because the necessity of job analysis, you know, what do they gain? Do they have to do it? Isn't a straight Yes, no. How high stakes is the role? How complicated is the role? How known is the role? Is it a new role? Arguably, roles are newer and more confusing and more complex than they have been with the kind of pace of organisational change. We're defining new jobs almost every day, whereas if we perhaps look at the retail and hospitality sectors, it's not that we don't understand what those people do every day and what we need to hire for. There, the job analysis, I think, is playing an even more important role in reducing bias. So, although hiring managers will typically have a really strong predisposition for what they want to hire for, there's going to be layers of human bias all bundled up in that. So, by going through the robust job analysis process and asking, what do these people actually do and what do they actually need to perform it, you pull out all of that bias. And so, the outcome for an organisation is a less biased hiring process and employees who will stay longer in the role, perform better, and be happier.

Kate YoungWhich is solving an expensive problem, isn't it? I think I've certainly worked with with clients who have been frustrated that they're hiring people in, they only stay a few months and they, and they move on. But investing this time upfront does feel like a lot, particularly when, well, everything right now, isn't it? The climate that we're in, the state of the world, the state of the economy. Is this where AI can come in and make it more time effective, make it more cost effective? Is that something that— Sapia is doing?

Kate Young100%. It is absolutely hugely time-consuming. I'll share with you my kind of favorite job analysis story from days gone by. It was pre-pandemic, so probably about 2016. My alarm had failed to go off that morning, which was helpful when catching a 6:00 AM flight to Munich. So, I was rushed onto the plane to catch a flight to Germany to go and work in the patent office there. to identify what makes a really great patent officer. This is a fascinating role to work with, actually, because in patent offices, you need people who are highly, highly specialised in their role. They're deep, deep technical experts, very, very well qualified, but they're also really happy to sit in a room all day on their own reading documents. So, to try and identify the type of people who are going to excel in that role, who aren't going to leave because they're bored, or in fact, they had issues with depression and anxiety in that role, is really critical. But the exercise itself was immensely painful, even setting aside my lack of sleep and airport panic, because I was in a room for 8 hours, or it felt like 8 hours, it may only have been 4 or 5, with 6 or 7 patent officers. And there were another 20 patent officers that had been beamed into this presence technology, which was fancy advanced video conferencing. So I was trying to corral 20 to 30 highly paid stakeholders into— we had cards, and the cards had qualities on them, and you kind of had to swap the cards in and out according to how important you thought the qualities for the job were. So, trying to get 20 or 30 people to coalesce around that, who didn't really understand what the qualities on the cards were in the first place, had very different opinions. And kind of to my earlier point with hiring managers always believing they know the job, had their individual biases, was a difficult exercise that I'm not really sure got us to a good outcome. And I think the reason for that, the reasons for that, many, too many stakeholders, when you do a traditional job analysis process, suddenly everybody hears it's going on, focus group interview, I want to go, I've got an opinion, I want to be involved. And there's no kind of structure. It's hard to bring in structural discipline to the process when you are facilitating all those human conversations. Whereas what AI does is rather than having to lock 25 people in a room for 6 or 7 hours or doing lots of focus groups or interviews over a series of days, is it it condenses that process through bringing in data at the start. So, what Sapia have done is built a tool called Job Analyser Studio. Now, this tool sits on top of a competency framework. For listeners who may not know, a competency framework is a list of, we can also call it skills, soft skills. So, things like planning, negotiating, strategic thinking, learning agility, those types of soft skills that are often in the workplace. We have 25 of them. which are well-defined, because we analyzed 37,000 job descriptions to get to them. So, the tool knows the relationship between tasks performed on a job and the behaviors or the competencies required to undertake those well. So, what it can do is consume a job description to make an initial suggestion, which is data-driven. These are the behaviors that are likely to lead to success in that job. And then that's when you get into the question of where do you bring humans into that process? And because we've basically done the first 90% of the work, through using AI to support doing all that work that I otherwise would've been shifting cards around a room. You get something to review and agree with, disagree with, or challenge. So, your conversation becomes incredibly productive because you're just doing the parts that a human really needs to review. So, I can get through that process in about 90 minutes now, which is much lower use of organisational resource. And of course, it can be fully remote on Zoom, time zone agnostic. So, I believe it's a massive step forward in job analysis to be able to use AI.

Kate YoungSo, just so I understand the process, you You'd bring a job description to the AI application, and then that job description would be analysed and would pull out the competencies that it has learned is most effective or most predictive of future job performance?

Kate YoungEffectively, yes. There's a bit more in that, so I will spend time upfront asking questions beyond a job description, and job descriptions really vary in quality. If you've ever seen any or even had to write any, they can be excellent and they can be awful. So, if there's any gaps in it, we need to spend time plugging those. And we have a conversation about, typically do a kind of, it's like the old rep grid technique, which is where you'd say, think of 2 employees, one who you really rate and one who you don't. I make it that simple for people. And I say, tell me why, what John does better than Barry in this situation. And we'll go through some quite simple probing. But just going through some very simple questions in about 20 or 30 minutes added to the job description gets you enough information for the AI to make a really congruent recommendation.

Kate YoungAnd in terms of the development of the AI, because things that I have heard or read or seen people talk about on LinkedIn, seen people talk about, you know what I mean, is things like, well, AI is biased because it's learning from information that's already biased. So, how do you And Sapia ensure that, as an AI application, it is supporting ethics and trying to remove bias from that part of the process?

Kate YoungSo, in the case of job analysis, there's a few different steps. With AI, any AI system, not just limited to hiring, it comes back to the data you put in. Garbage in, garbage out. I think we've all heard that phrase. So, the job descriptions were very much cleaned, curated, and interrogated for bias that we fed into the system. So, and we can all, I mean, I've seen job descriptions that say things like, talented in all areas of life, may demonstrate this through sports outside of work, which is horrifically ableist language, in fact. Or they'll say words like dynamic or highly go-getting, words that would tend to lean more towards how majority groups tend to show up in the workplace. We cleaned up all of that before ingesting these job descriptions. So, the data itself was as clean and unbiased as it possibly could be. And then the other way is this human in the loop. So, once we've ingested all the information, we sit there and we review it and we interrogate it for bias. So, a good example, one we always get to, is extroversion. So, people who are energetic and chatty, they do well in lots of roles. And they're also more likable. And I say that as an introvert. Extroverts tend to sort of, we tend to be drawn to them more. And so, a really interesting flag is when kind of extroversion comes through as something that we want to measure against, because then I'm kind of going to turn around and ask the question and say, well, does this person actually need to be giving out energy to others all day? And if they're on a shop floor selling cosmetics, for example, yes, they absolutely do. They're sitting in an office, Then what would probably happen is we've got a little kind of bias towards perhaps likability that's crept in there. So, there's that interrogational layer as well. So, it's really the data you put in and the interrogation layer that pull everything out at the job analysis stage.

Kate YoungI'm a client of yours. I've gone through this process with you, created the job description. We've gone through the— Sapia has done its thing. It's popped out the competencies that we need. What happens next? What do I do with that?

Kate YoungSo, next there's another stage. So, we've got the competencies. Then we make questions. And that's an equally critical part because you have to measure what matters, which is the right competencies or skills for the role, whatever language we prefer. But then you have to ask the right questions that are going to feel fair and inclusive and give everybody the right opportunity to demonstrate their skills. So, the AI enables this in a very clear way because it consumes the competencies we've agreed on. We've chatted about, we've swapped Swapped in, swapped some out, we're all happy. And then it's going to combine that data with the job description that is really clear about what's required in the job to come up with some really tailored questions that are explicitly linked to the competencies. And it's also trained to do really neat things like balance the questions. So, from an inclusion perspective, for making hiring fair, we know that some types of candidates respond badly to questions about, imagine it's your first day in the job and the till has broken. Some people, don't like to do that kind of abstraction. They can't do it. Whereas if we ask questions, tell me about a time when you had a leadership role, for example, we know that candidates who are more advantaged in life are more likely to be able to share those examples. So, the AI is quite clever, and it balances those types of questions as well in its final recommendation. So, all that to say, in terms of what happened next, you get your questions, and then those questions are pushed into this chat interview product that we have. And the chat interview is these 5 questions we've come up with, with the mix and the balance, that candidates interact with via their laptop. tablet, phone. They answer the questions in their own time. So, there'll be preamble, welcome to your interview with such and such, and, you know, we wish you luck, and please be yourself. And there's little prompts as you go through, like, oh, Leanne, you've done 3 questions, you've only got 2 more to go, you know, keep going. And there's kind of a thank you and feedback at the end.

Kate YoungSo, this is a chat— I don't wanna say bot— application that is leading this interview with candidates?

Kate YoungSo, we're really clear it's not a chatbot. And the reason I make that distinction is chatbots are normally dynamic. In that if I say something, the chatbot's going to give a different response based on what I said. And that might be scripted, or it might be based on a large language model. So, it might be a bit cleverer than that and alter its response, like having a conversation with ChatGPT. Ours doesn't do that. It gives the same questions to every candidate. And the reason is because we are very passionate about fairness and inclusion in hiring. And one of the best ways to achieve that is to make sure everyone gets the same questions. And it's our view that Conversational AI, that dynamic AI, hasn't reached the standard yet where we would be happy to deploy it to candidates. So, we keep it all the same for everyone.

Kate YoungBut it is an automated system that's presenting the questions to candidates?

Kate YoungYes.

Kate YoungAnd candidates presumably are aware of this, that it's a— of what's happening in that process?

Kate YoungYes, yes. They log in and they'll get an interview, some text explaining, and then they'll see the first question appear. And it appears like your message does on your iPhone. So, you see the 3 dots, and then the message loads. So, it's very clear it's not a human being, it's an automated digital experience.

Kate YoungAnd is this typically the first stage of the recruitment process?

Kate YoungYes, we typically are. We say it's kind of top of funnel. So, we usually look to replace either the CV. We don't like CVs. They're horrible ways of predicting how someone will perform in a job. They're riddled with bias. So, where we can, we like to replace the CV and bring the chat interview in at that stage.

Kate YoungSo, if you are not looking at CVs, how do candidates get shortlisted for this initial interview stage?

Kate YoungThey don't need to be shortlisted. So, there's a great strapline of my CEO's is, everyone gets an interview. So, you apply and you move straight through to the chat interview, and then the shortlisting happens after that based on your performance in the chat interview.

Kate YoungI'm, oh, I'm having all sorts of thoughts on that. I, I would imagine as a candidate, that's a really positive experience to go through because it's so soul-destroying, isn't it? When you're applying for so many roles and never hear back and never get any further and never get the chance to even demonstrate what, what you're capable of.

Leanne ElliottI like that.

Kate YoungAnd are you finding that candidates are responding well to that?

Kate YoungIt's incredibly positive. Um, so our candidate satisfaction is around 9.10. Across all clients. We get huge— we ask for free text feedback. We ask people to write a comment. We just did a massive analysis. I say we, the data science team did it. I didn't do it because they took thousands of words of text and went through them to pull out themes, and they were overwhelmingly positive. I think the one that always stands out for me is candidates get a chance to feel they can tell their story because it's untimed, because they're typing. And because they weren't, their CV wasn't rejected because a CV is a list of things we did. It's not our story. It's not our words. It's not us reflecting on our experience. So, it's incredibly positive for candidates and clients do report that pretty much universally.

Kate YoungAnd candidate experience is not something that is thought of very often or scores very highly for many, many organisations, is it? That's incredible, that rate of candidate satisfaction you're getting.

Kate YoungIt's exceptional. And something I realized coming into this job, because I think I'm often accused of being almost too much of a scientist at times and being obsessed with the science of solutions. So, I kind of used to underrate candidate experience, but what I was really forced to reflect upon in this role is candidate experience is candidate trust. And if candidates don't trust a hiring experience, they're not going to engage with it fully. Give full responses, show their true selves, and be able to kind of demonstrate their skills to the best of their potential. So, the experience itself is making the hiring decision more robust because candidates are fully engaged with it.

Leanne ElliottDid you know that the UK's number one management podcast, that's us, by the way, and the UK's number one marketing podcast are both on the same podcast network?

Al ElliottWe actually have a lot in common.

Leanne ElliottWe both use And we both had to wrangle Rory Sutherland on an episode.

Kate YoungAnd 2 out of 3 of us are devilishly good looking.

Al ElliottAnd you're not going to say which. Fel, host of Nudge, UK's number 1 marketing podcast. It's brought to you— we need to do this in 3s— the HubSpot Podcast Network.

Leanne ElliottThe audio destination for business professionals.

Kate YoungSeamless.

Al ElliottSeamless. Tell us about your latest episode. Well, it's called Truth, Lies and We've Work.

Kate Youngjust done an episode on fake fandom and how New York indie bands are paying agencies to create fake TikTok videos about how much they like their work. And we talk about the behavioural science behind fandom and how that encourages people to enjoy the music and all of that good stuff, and do some big things about how this affects the world of politics, business, and brands as well.

Leanne ElliottIt is, it's such, it's such a good show. Of course, you'll hear all the stuff you want to hear about how to grow your business by using behavioural science in your marketing.

Kate Youngmarketing, but there's also just some really interesting episodes that'll be right up your street.

Leanne ElliottPersonally, I enjoyed Can Balsamic Vinegar Make Beer Taste Better?

Kate YoungIt can.

Leanne ElliottAnd Are We All Just Status-Seeking Monkeys?

Al ElliottI am. Go and listen to Nudge wherever you get your podcasts.

Kate YoungBut come back.

Al ElliottYeah, come back.

Kate YoungDefinitely come back, because Nudge isn't as good as this, so come back.

Al ElliottI'll cut that out. Keep that in.

Kate YoungYeah. And again, as well, as you say, in terms of fairness, that type of process allows for people to think and reflect, doesn't it? It doesn't put time pressure on it, which, yeah, I'm thinking of my nephew is dyspraxic and he is incredibly intelligent and empathetic and talented, but he, you need to give him space to communicate that. So, it's, I can imagine that would work really well for neurodiverse candidates as well.

Kate YoungIt really does. And it's something I'm actually really proud of. My own daughter was diagnosed as autistic last year. So, I think a lot about how she'll experience applying for roles. And we track this data. We're data obsessed, as you might expect from an AI company. And we find our satisfaction rates for either neurodiverse candidates or candidates who identify as having another disability typically comparable, maybe 0.1 smaller sometimes, just because hiring's always going to feel that little bit harder.

Kate YoungYeah.

Kate YoungFor neurodiverse candidates, but they're more or less comparable and, and really positive feedback themes.

Kate YoungThat's exceptional.

Kate YoungWow. It is.

Kate YoungWow. Okay. So, I'm thinking now, 'cause I can hear Al in my ear as like the business owner going, mm, they asked this, think about this. I imagine he'd be going, he'd be sitting there going, wait a minute, candidates all go through to an interview that they can type or speak into with no time limit. How do I know that's them? How do I know they've not got their mate or their parent filling it out? How do I know they're not asking ChatGPT? How do you check for the authenticity of who it is you're actually interviewing?

Kate YoungIt's a really important question, and it's a question that's not unique to an AI chat interview. We've had, call it unpropped, unsupervised hiring methods for years, personality tests, cognitive ability tests, those types of things that candidates will do. with no supervision whatsoever, and we've always just had to trust it's them. And unless we're going to proctor to supervise every candidate, which given the volume of applicants to jobs now is impossible, we have to accept, I believe we absolutely have to accept some level of risks. There are proctoring solutions out there where you switch on your camera and you're being kind of watched. It's not something we are willing to explore because We value that trust with the candidate and that inclusion, and being filmed by camera is the opposite of an inclusive and comfortable experience. So, there's a little bit of a trade-off there. So, we're choosing to prioritize that candidate experience and trust over absolutely knowing it's not their mum doing it. There's a lot of prompts and nudges in there around making sure it is them. So, if we detect they're using AI-generated content, we stop them copying and pasting anyway. So, if they're going to use ChatGPT, they've then got to type it all over, and you would just— that friction just will discourage a lot of people. And then, if we detect they're doing it, they get a nudge like, Look, you know, we think you might be using AI content. Please don't do that. We need to see your responses and your truth. So, and if they do it again, they get prompted again. Typically, most people stop after that prompt. They don't do it. So, yeah, I can't do anything if they get their mum to do it. But by and large, because they enjoy the experience and they trust it, I think those rates are going to be pretty low compared to other solutions.

Kate YoungOkay. So, what happens Next, I'm really enjoying this kind of going through the journey, like, as a candidate and thinking like as, as the, the client as well. So, I've done this interview. What happens then with the data? What happens with the client? How do they get notified? What, what, how did, what happens next?

Kate YoungOh, well, how nerdy do you want me to get?

Kate YoungVery, please.

Kate YoungVery nerdy. Okay. So, in terms of data, so, we, the data is their 5 chat interview responses. And this is really, really important because when we talk about AI in hiring, people will go to certain well-known lawsuits, and they'll assume that, for example, my name, Kate, which is clearly a kind of female-leaning name, is going to go in there and it's going to tell the AI algorithm that I'm a woman, and that's going to bias it. So, we are super, super clear that the data is those 5 responses. They're consumed by a scoring model. So, it's a scoring model, you can call it like a prompting framework. And I kind of like to think of this as Something that's holding the hand of your large language model. So, we use Claude. All your listeners will have heard of ChatGPT. That's the most popular one. Claude's just like another one. It has the best privacy settings. That's why we choose Claude and some other things. And it's holding the hand of Claude and saying, okay, Claude, we need to score this candidate's response. Imagine you are an expert occupational psychologist trained in all these principles. This is all the things I need you to consider. This is what you're scoring against. This is how you check you're getting it right. If you get it wrong, this is how you correct yourself. This is what counts as evidence. This is what doesn't count as evidence. And it'll go through that, and it'll make you do it 3 times. And then it's going to give scores. So, you get— I mentioned these competencies before. So, you get a 1 to 5 score for each competency. And then they're pulled through into reports. So, more importantly than the score, and the score is what will be used to help effectively rank the candidates, so hiring managers can start to identify the ones they want to move through to whatever the next stage is, typically a telephone or face-to-face interview. But what you're getting is one of those scores, a lot of explanations. So, when we think about AI hiring and AI governance, explainability is the beginning, the middle, and the end. Where you get a lot of fear in AI hiring is what we call black boxes. So, data in, something comes out, and no one understands how it got there. Of course, I'm not going to trust that. Why would I? Whereas with our model, the hiring manager will get a report that explains exactly how that 4 out of 5 or whatever's arrived at. So, I'm giving Leanne a 4 out of 5 for teamwork because she's told me an example of how she motivated a team when they were under pressure. And in question 3, she told me about how she comforted a team member in a specific scenario, and it will connect A to B repeatedly. So, the hiring managers can have real trust in that. And also, they can query it if they want to as well, because nothing's hidden.

Kate YoungAnd what's the threshold to pass as such in terms of the next stage? Is that individualised by the client or the role, or is it kind of like a set benchmark?

Kate YoungWe don't do automated pass/fail. We give back a score to clients, and it's depending how many people they need to move through to the next stage, we'll define where they tend to set their benchmark. But typically, because of the types of roles we're hiring for, the client's going to be applying a couple of other filters as well. So, if we're thinking about retail, they're going to also just do some filtering for people who are available to work Saturdays, and so on and so forth. They meet some other criteria. So, they're going to do some filtering, and they're going to rank the candidates, and they're going to work through those from kind of top to bottom and start moving them through to interview, but the actual number they'll move through from kind of that first stage to second stage varies massively across client, just depending on their hiring needs.

Kate YoungSo, I'm also seeing at this point, hiring managers are quite happy because the first that I am, obviously I've been involved in the job analysis and checking those competencies, but the first that I am then, in terms of my time, dealing with is candidates that have been screened, that I've got lots of nice information for, that I can filter for different criteria that I want, and then I go on to, you say, the face-to-face, the telephone interview stages. Is that what you're hearing, or are hiring managers a little bit distrustful of the AI that's happened and decisions that have been made without them?

Kate YoungWell, hiring managers are not a homogenous group. It would make my job a lot easier if they were. By and large, hiring managers are delighted because their time to hire, which is what most of them are targeted on, is going down as much as 70 to 80%, especially for high-volume clients. And they are finally released from the tyranny of CV screening, which nobody enjoys. There's a little bit of novelty when you first start. My first role was as a recruiter. I was terrible at it because there was too much sales involved. And I used to find it really fun going through every CV and going, huh, that person's put a picture on. That's ridiculous. But then the novelty kind of wears off because you're just trying to make decisions with rubbish data and repetitive data. So, they're released from that. Their time to hire improves. Their quality of hire improves. But Yeah, there are absolutely people who are reluctant to trust, and then that becomes a job for us to run the analysis, prove it's working, and kind of hold the client's hand through that change management piece. And typically, where we struggle most is where there's an easy alternative for the hiring manager. So, again, in something like shops or retail, walk-ins, where people walk in and say, I really want to work here, that's really compelling for a hiring manager, like somebody's bothered to travel to the store, make themselves smart, walk up, shake their hand, and say, I really want to work here. Can I interview? So, that's usually our kind of big tension point is pulling people away from that and into the AI hiring tool.

Kate YoungWhat's to stop a hiring manager filtering for, I'd only want to employ women, I only want to employ men, I don't want anybody who's neurodiverse, I don't want— you know, how do you— is there any way to prevent that? Or once that data is back with the client, it's theirs to do with?

Kate YoungSo, none of that data exists as filterable in our system. We collect, are you male, female, what's your ethnic group, only for anonymized background monitoring, so we can check the system's fair. Ultimately, I can't stop a hiring manager if they've met 2 candidates, and one's a man and one's a woman, and they really wanted to hire a woman. I can't stop them making that decision, and nor should I really, because the decision— the tool doesn't exist to make the decision, it exists to provide the best possible information. So, the decision sits with the hiring manager, and that's another thing that we're really keen on. the kind of really important principle around AI in hiring and accountability. You've got to be really clear what decisions sit with the humans, what decisions sit with the AI, what sit with me as the psychologist, what sit with the hiring manager as the person doing the job.

Kate YoungAnd if I'm recruiting for a role, and I know this is going to be a massive depends, but let's say I'm hiring for a role that typically in traditional recruitment might get 500 applications. Through this system, how many candidates am I likely to pop out with at the end, where then we'd take them to kind of second stage of that recruitment process?

Kate YoungOh, that is a massive it depends, because—

Kate YoungI guess what I'm trying to get at is, do I— how is this going to change what businesses or hiring managers might be used to? Am I getting far less candidates through? Am I getting maybe 4 or 5? Am I getting hundreds that I need to, to go through? And I understand it's going to be Depending on the role in the— but I guess I'm wondering if there's a typical kind of outcome that hiring managers are going to get when they're faced with people who've gone through that competency interview.

Kate YoungSure. So, if you want a typical outcome, we generally, we do sort of give a yes, maybe, no label based, and that's purely based on score. And that is to help busy hiring managers if they're making decisions at pace. And if they've already established their trust in the tools, they're like, you know what, I know Sapia assesses well, I know the score's robust, I'm going to trust the the tool, I'm going to interview the people Sapia recommends. We'd usually look to be about 30 to 40% yeses. So, we'd be saying, you should probably review these people now. A lot of hiring managers won't do that. They'll go top down. They'll start with the best scores and just work down. But yeah, that's typically the kind of reduction rate.

Kate YoungAnd in terms of, again, going back to that candidate side of it and the experience, will all candidates get notified if they're not successful? How does that work in terms of the hiring manager's role versus Sapia's role?

Kate YoungSo, that's not Sapia's role because the decision, it doesn't sit with us. So, what Sapia will do for candidates is everybody will get feedback. So, it's a really nice, it's called My Insights. It's actually a lovely report because it's very kind of coach-based, very conversational, quite informal, and it aims to really give the candidate a sense of their strengths. But also a tip at the end. So, when thinking about future job roles, maybe consider structuring your day or something like that. So, there's never that ghosting. Sapia makes socks. They're lovely socks. They're very warm. If you see them at a conference, do grab a pair. And one of them just says, no ghosting, on the foot of it. So, they never get ghosted. They always get feedback from the Sapia side. But in terms of how clients choose to notify candidates, That very much sits with them. That said, we are really passionate about candidate experience, and we work with clients to set these experiences up. So, typically, we will have that conversation with them. How are you planning to communicate to candidates? What's your messaging? And offer them that support.

Kate YoungWho are the clients that you are primarily working with at the moment? Are they larger businesses?

Kate YoungYeah. So, we sell to enterprise, and the reason for that is it is an AI hiring solution. So, it's kind of a governance and a setup layer. Which a small business isn't going to have the architecture for yet. Maybe in the future when this is more kind of normalized and we're able to kind of democratize the tech further. So, for example, we work with Holland Barrett in the UK, which we all know where we get our vitamins and our tasty, healthy protein bars. They're a really good example. Also, Costa Coffee. Do you know, I'm not a salesperson, so I always forget who I'm allowed to mention and not, but those are 2 very strong examples in the UK. And then within Australia, Australia, um, so actually headquartered out of Australia, with a very strong relationship with Kmart, which are their sort of main big kind of retail store. They were recently featured on the Josh Berson podcast talking about how they'd increased the efficiency of the experience with Sapia.

Kate YoungAnd I guess what— I'll be honest, Kate, right? As you're going through the process, I'm loving it. It's everything that I was taught and everything that I've done with clients. And I'm also thinking, well, I'm redundant then, because if AI can do all that, they're not going to need an occupational psychologist to do it. So, now I'm thinking, is there a future where Sapia would maybe, I don't know, license this software to other practitioners to use with clients? If we get to the point where small businesses have that structure, what's the future with it in terms of its growth and application?

Kate YoungI'm not sure we've been asked the licensing question. I can't see why we wouldn't. But I'd also want to say it's definitely not making us redundant. I did a workshop last week at the Division of Occupational Psychology Conference over in Cardiff. It's a beautiful city, wonderful conference. And the workshop explored what is our role as psychologists within an AI hiring system. And really, the takeaway is we're just moving from individualized, hands-on judgment and ad hoc review to a very sort of systemized role. So, we're designing the architecture the governance, the parameters that need to sit within a system to make sure it's working well and it's fair and it's unbiased. So, it's like many things. Is AI going to make me redundant? Absolutely not. It's just going to make your job more interesting because you can do more of that elevated, interesting problem solving. What is the future? So, I think for us, what we want to do is take everything we're learning through chat interviews and or the intelligence we're building around competencies and use that to leverage it more from a talent intelligence perspective. So, not just for hiring, but also to support things like internal mobility. So, where you're kind of collecting all of this rich data upfront about someone, make it work for them as they travel through the organisation when you're thinking about internal roles or perhaps coaching, so on and so forth. So, doing more with the data we have because it is so rich.

Kate YoungThere might be someone listening who goes, but you're reducing somebody to a number. You're giving them a score. You're giving them a 3 or a 4. Is that the right thing to do? Is that dehumanising somebody?

Kate YoungIt's a really fair question, and it might be a kind of overly pragmatic response, but I would always encourage anyone asking that question to say, what would you prefer? By which I mean, is it better we reduce someone to a CV that's screened in 20 seconds, and you get a decision based on bad data, bored recruiter? Yes, they are given a score at the end of the interview, and yes, it is used to make a recommendation as to whether they move forward. But that score is based out of all of the robustness that I've just described, all of the great questions, all of the great principles. So, ultimately, when you've got thousands of people in a hiring funnel, you need a piece of data to make a decision on, because there has never existed a time where all those thousands of people would've got a lovely face-to-face interview. That's just a kind of nirvana we like to imagine that was actually never real. We used to post off CVs and hear nothing back, and it was heartbreaking because, you know, you put it in the envelope, you put the stamp, and gone to the postbox, and you got nothing back. So, yes, it's just, it is a number, essentially, but it's a robust and fair number. And I think in a big, high-volume situation, that is the very best we can do for people.

Kate YoungIn terms of Any small business listening who's like, this sounds really cool, but clearly it's not something that I can— is accessible for me right now. What advice would you give them in terms of making their recruitment process more robust, more fair?

Kate YoungWell, the best advice for a small business is find a friendly psychologist in your network. There's a lot of really great freelancers out there who will happily— and the thing about psychologists, I'm sure you've experienced this, Leanne, we just care so much about getting it right and being fair. So, most will just give 30 minutes, an hour. It's a bit like your free appointment with the solicitor everyone always talks about. It's kind of free. Most psychologists will give that little bit of advice. And it's kind of the Pareto principle. You can almost get 80% of the gain with 20% of the effort just by doing even rudimentary job analysis, just simply prompting people to sit down and go, what does this person actually do? Think. Don't think about who you like. Think about what they do. And then looking at your hiring process and going, are we giving the same interview to everyone in the same role, or are we just going off the cuff? And have we space to maybe introduce something a bit more structured here? And then also with the final decision-making process, how are we making that? Who's in the room? And how, again, how are we making sure it isn't the loudest person wins? Because we as psychologists call them wash-ups. You get everyone in the room and you say, oh, I like this, and I rated this, and I did that. And actually what happens there is just that the loudest voice typically makes the decision. So, there's some real low-hanging fruit that local friendly psychologists can probably direct you to with minimum effort. But failing that, go to Sapia.ai. We've made lots of white papers with some really good advice on there. They're definitely worth reading. There's good analysis in there, but also some really nice takeaway commentary. And there's lots of free webinars and resources around. Sapia do loads. I did one on inclusive hiring the other week, and they're a good free resource just to start educating yourself a little bit.

Kate YoungSay there's a business owner listening who goes, this all sounds very nice, Kate and Leanne, And sure, I could get in touch and find a friendly psychologist, but you've already told me how this is really manual and really expensive and takes ages, and I don't have the time for that. I need to get somebody into my business quickly. What if I just send all the CVs, put them all into ChatGPT or Claude, ask them to pick out the one that matches the job description best, and I'll interview that one and hire that one? Could I do that?

Kate YoungYou, you, I mean, I'd say you absolutely can do that, but the cost of doing that is so much more. than taking a beat and thinking about putting a little bit more structure and science in your process, because the cost of a bad hire, especially for a small business, where they're going to be a greater percentage of your workforce, is absolutely massive. So, it is absolutely worth taking a beat, upskilling yourself a little bit, thinking about some structure, thinking about some basic points of measuring what matters, and just interrogating your decision-making process. And so, there's loads of easy how-to guides and basics out there. Definitely go to Sapia, but there's also plenty on the rest of the internet. It's not hard to start getting some of the basics right, but it's when you kind of enter the volume hiring space that the stakes then become really high, and that's when something like Sapia is going to get involved.

Kate YoungI think this is really cool, and I think there's also organisations out there that are the HR-based organisations, recruitment-based organisations that might feel slightly threatened by this, that are in high-volume recruitment, who might make arguments that human-first is better, it's more personal. It's— how do these things coexist in a world, or do you think that high volume is going to become AI first?

Kate YoungI don't think there are any high volume processes that are still human first. No one's got the— absolutely no one has got the resources. If they are, they're the clients I've spoken to in the last year who are saying, my recruiters are spending hours, hours, hundreds of hours a week screening CVs. This has to stop. This has to stop. We can't afford it, and it's not scalable. So, I kind of think in terms of putting some level of automation in, whether it's AI or not, but some kind of standardization at the top of funnel has kind of sailed, that ship has sailed. If you ask if the two can coexist, absolutely, and they absolutely should. I think AI in hiring is most effective at the start, at the top of the funnel, to help you when you've got a lot of data, to synthesize that data. That's what it's really doing, is taking all the data, And synthesizing it into something consumable for hiring managers. And then at that point, the humans come in and do those face-to-face interviews. And that could be solutions from more traditional HR providers. There are loads of wonderful providers out there who provide online role plays or other types of exercises, and those things can absolutely coexist. And also, as a psychologist, I love it when the next part of the hiring process is really different to the first part, because the chances are it's going to measure different things, as long as they're still relevant things, And you can get more information on the candidate, so everyone wins.

Kate YoungI have a very nerdy question.

Leanne ElliottI love it.

Kate YoungSo, you mentioned before that you gave the model all of the data from job descriptions, it popped out. How many competencies did you say?

Kate YoungIt was 25.

Kate Young25. Are you looking at, maybe you already have, are you looking at, because I know places like SHL have like the standard Grade 8 competencies, are you looking at potentially, I don't know, using these 25 competencies in a way that could support smaller businesses, for example? Is that what— yeah, tell me about that. I'm interested.

Kate YoungIt's a great question because the competency framework itself is a great resource. And if you compare it to something like the Great 8, or Korn Ferry's comp framework, or any other really established ones, you'll actually notice quite a lot of commonality. And that's because those frameworks, e.g., the Great 8, were also created in a really robust way. Now, they didn't use AI, they did it long So, they did a relatively similar process, but manually. So, less data, more time, but it was not dissimilar. So, when you have robust psychologists working with similar data, you will get outputs that are somewhat overlapping. I think we could absolutely use that competency framework to support small businesses. So, one thing we've been experimenting with a little bit is just using the Job Analyzer tool to just help people define their face-to-face questions. So, they're not going to bring in a chat interview or AI. That's not going to work for them because of the AI governance piece. piece and the scale, but you can absolutely use the tool just to support defining your face-to-face, your first-stage interviews, and that's a massive time save potentially for small businesses.

Kate YoungIs this public knowledge at the minute in terms of what these 25 are? Is it still kind of proprietary information? Where's that all standing?

Kate YoungI'm pretty sure there must be a write-up of them on our website, yes.

Kate YoungAh, okay.

Kate YoungI should be able to name the document, but I confess I can't. But yes, it's not a secret, certainly.

Kate YoungIf I'm listening as a potential client, how I look into if I could use this in my business?

Kate YoungDefinitely go to Sapia.ai. All the resources are on there. There should be a contact form on there. Anyone listening who wants to talk about the science, they're welcome to contact me either via LinkedIn. I'm Kate Young at Sapia on there. I'm not hard to find. I'm quite noisy on LinkedIn, or kate@sapia.ai. But we love talking about this stuff and we're pretty open about it. So, absolutely get in touch. Amazing.

Kate YoungAnd if listeners take away just one thing in terms of hiring better, what should it be?

Kate YoungI think I said it a few times, but it's measure what matters, because unless you define what you actually need to be successful in that role, it doesn't matter if you choose the best tool, the worst tool, how you execute your process, you've undone yourself from the start. So, even if you can just apply some very rudimentary job analysis, that's already going to get you a big way That was Kate Young from Sapia.ai.

Al ElliottAnd I'll be honest, I did come into this conversation, like Leanne, slightly skeptical about AI in hiring. I love AI, but hiring felt like the one area where humans should probably be involved in every single stage. But now I'm starting to think we've been doing it wrong for a very long time.

Kate YoungI'm the same.

Leanne ElliottBut what I learned was that AI in this context is acting as an amplifier. So, if the science and the governance is right, then AI is simply making the process faster, easier and fairer. And Kate's point was very valid. When done right, with the support of occupational psychologists, this actually makes for a much better candidate experience. And that's something I've been pushing for for a very long time.

Al ElliottSo, these are the 3 takeaways for leaders thinking about how they hire.

Kate YoungLeigh?

Leanne ElliottNumber 1, your hiring process is only as good as your job analysis. I'll say that again, it bears repeating. Your hiring process is only as good as your job analysis. Before you worry about which tool to use, AI or otherwise, ask yourself, do we actually know what we're hiring for? Not the job title, not the vague list of requirements on the job description. What does this person actually do all day, and what does it take for them to do it well? If you can't answer that clearly, no tool in the world will save you.

Al ElliottYes. And lesson 2, bad hiring isn't just expensive, It's unfair. Kate made this point that we've romanticized the old way. The reality was that CVs were screened in 20 seconds by someone who'd lost enthusiasm for the pile by number 6. At least AI, when it's done properly, applies the same rigor to every single candidate. If you're a small business and full AI hiring isn't accessible yet, even a 30-minute conversation with a friendly occupational psychologist— hint, hint— My co-host is one of those, can get you 80% of the gain for just 20% of the effort.

Leanne ElliottAnd lesson 3, the loudest voice in the room is probably the least reliable. So whether it's a hiring manager who just knows who they want, or someone dominating the debrief after an interview, Kate's research is pretty clear. Unstructured decisions default to bias. You don't need an AI system to fix that. You just need some structure, some agreed criteria, And a process that stops the loudest voice from winning by default. You can find Kate and the team at sapia.ai. There are brilliant white papers on there, free webinars, and a contact form if you want to explore whether it's right for your business. Kate is also very active on LinkedIn. Search Kate Young at Sapia and you'll find her. All the links, as always, are in the show notes.

Al ElliottThat's all for today. So go and write down what your next What AI actually needs to do, not just be. This is Truth, Lies and Work. We'll see you next week.