A resume used to tell you something. So did a well-written cover letter, a thoughtful take-home assignment and a confident answer to “tell me about a time when…”. Each one took effort, and the effort itself was a signal.
That’s no longer true. Generative AI can polish any resume, draft any cover letter and rehearse any interview answer in minutes. Candidates aren’t doing anything wrong by using it – most employers expect them to use AI on the job. But it leaves hiring teams with an uncomfortable question: if everyone can sound brilliant, what are we actually measuring?
We put that question to Tracy St.Dic, VP Global Talent at Zapier, on The CHRO Mindset Podcast. Her answer wasn’t to ban AI from hiring, or to hand hiring over to it. It was to be far more deliberate about where evidence comes from, and who makes the call.
Start by deciding what AI is allowed to decide
Before choosing any tool, Tracy recommends drawing one clear line: AI gathers and organizes information, evidence and data, and humans make decisions.
In practice, that means AI can take on more of the work of summarizing, pulling data together and organizing it – scheduling, first-pass screens, interview prep and scorecard summaries. Anything that affects someone’s livelihood – who advances, who gets an offer, how you close a candidate – stays with a person.
The line matters because it answers most of the hard questions before they come up. When a vendor offers to automate decisions, or someone on the team wants a tool to auto-reject candidates, the answer is already written down.
Stop testing for output. Test for judgment.
When AI can produce a strong answer for anyone, the answer itself stops being the signal. What still separates candidates is judgment: how they frame the problem, how they use the tools, whether they notice when something is wrong, and whether they own the result.
That’s why Tracy argues that if AI is part of the job, it should be part of the interview. Instead of pretending candidates won’t use it, her team publishes guidance on how to use (and not use) AI in interviews, shares sample prompts and asks candidates to sign a statement of integrity. Skills tests have moved almost entirely to live formats, where an interviewer can introduce a curveball and watch how someone thinks on their feet.
Assess AI fluency as a way of working, not a tool list
“Uses AI tools” on a resume says very little. Tracy’s team assesses AI fluency through four lenses: mindset (do they lean in and learn hard things quickly?), strategy (do they know where AI fits in their function?), building (do they work with AI as a thought partner and push it further?) and accountability (do they know when not to use it?).
Candidates are scored from unacceptable, to capable – the minimum bar – and beyond, to adoptive and transformative. Whatever framework you use, the point is consistency: every interviewer should mean the same thing when they say a candidate is “good with AI.”
Look for evidence outside the interview
If the interview is easier to polish, the evidence around it matters more. Tracy describes balancing internal and external signals: what a candidate says in the room, and what the people who worked with them say outside it. Reference checks, often treated as a final box-tick, become a real part of the evaluation – ideally with people who were actually on the projects a candidate describes.
It’s also worth being precise about integrity risk. Tracy separates “capital F” fraud, where someone isn’t who they say they are, from “lowercase f” fraud, where candidates embellish more than they used to. The first needs hard checks, such as flagging a stated location that doesn’t match an IP address or phone signal. The second is best handled with better questions and better references.
Expect bias to move, not disappear
It’s tempting to assume structured AI screening removes bias. It can reduce some of it: every candidate gets the same questions, with no interruptions and no visual judgment, and candidates often say it feels fairer. But Tracy is clear that AI can also amplify bias or simply move it – into how the rubric is written, which questions are asked and what the system is told to value.
The guardrails are unglamorous but essential. A human reviews every AI summary and makes every decision. Teams run bias tests and test groups, and check pass-through rates before and after each AI step. Used carefully, AI can even surface bias people miss – such as noticing when a candidate’s strongest advocate on a panel is also the person who referred them.
Use AI to make hiring more human, not less
The fear that AI makes recruiting transactional is real, and Tracy agrees it can – if teams outsource everything to it. Designed around candidate experience, though, AI can do things that were never possible at volume. Every candidate can get the same patient first conversation, whether they’re the first call of the day or the fifteenth. Every rejected candidate can get specific feedback drawn from interviewer scorecards, rather than a line about someone being “a better fit.”
Tracy’s standard for this is “consistent and personalized at scale.” It’s a useful test for any AI step you add: does it make the experience fairer and more personal, or just faster?
Hire recruiters for the slope, not the snapshot
As AI takes on more of the administrative work, the recruiter’s role shifts toward advising – reading motivation, building trust, interpreting data and helping leaders make the call. Tracy looks for human skills like judgment, curiosity and emotional intelligence, the ability to combine data with business acumen and, above all, how quickly someone learns.
With tools changing every few months, what a recruiter knows today matters less than how fast they’ll learn tomorrow. Her team calls it looking at the slope, not the snapshot.
Where to begin
For HR leaders who want to move quickly, Tracy’s advice is to resist starting with the most advanced tools. Write down the vision first – how you want hiring to run and how it should feel to candidates – then start with FAR tasks: frequent, annoying and repeatable. Candidate emails and scheduling are obvious first targets.
She also recommends automation before AI, and AI before anything agentic. In HR, simple automation is usually more reliable, more accurate, safer and cheaper. Small wins build confidence, and confidence builds the fluency your team will need for harder problems later.
Five questions to bring to your next TA review
- Have we written down which hiring decisions AI may never make?
- Which of our assessments still reward polished output over real judgment?
- How do our interviewers define “good with AI” – and do they all define it the same way?
- Do we check pass-through rates before and after every AI step we add?
- Which frequent, annoying, repeatable task could we automate this month?
The bottom line
Polish is no longer proof. The teams that hire well from here won’t be the ones with the most AI in their process, but the ones most deliberate about where it belongs and where it doesn’t.
And waiting isn’t neutral. As Tracy puts it, if you’re not working to upskill your team in AI fluency right now, at this point it’s sort of management malpractice. The reassuring part: you’re not behind, and it takes very little to start.