An AI app was the most human part of my job interviews
I was a couple months into my job search when I felt truly listened to for the first time. When I felt the strongest sense of humanity from the other side as I had in the entire process.
The wackiest part? <Record scratch> My most-human interlocutor was an AI app.
The experience was unexpectedly profound and clarifying. It illuminated why job interviews can be surprisingly impersonal, and how AI might help bring some humanity back — or at least dispense with the charade.
Practicing job interviews with an AI
This all happened while practicing for job interviews using Nadia, an AI coaching app from Valence, an employee development platform.
I used the Nadia webapp, and entered this prompt to start the session:
“Hi Nadia, can you help me role-play a job interview? I’d like you to ask me the most common behavioral interview questions. Just ask them one at a time. Give me a chance to respond. Give me feedback on the answer before you ask the next question. Thanks.”
The app asked questions like:
“Describe a situation where you had to provide feedback to someone who wasn’t meeting expectations. How did you approach it? Take a moment to consider your answer, and whenever you’re ready, go ahead and share your experience with me.”
I answered by speaking into my headset microphone. (There’s also a text entry option.) The app recorded my voice, quickly transcribed my answer, then gave feedback. For what it’s worth, while I had previously written and practiced STAR/PAR-style answers, I decided to answer more off-the-cuff rather than trying to recite my pat STAR answers.
To help illustrate why the experience was so profound, I’ll first share a few examples of my answers and the app’s responses. My answers are largely copy-pasted from the app transcriptions; I’ve added some retroactive punctuation, paragraph breaks, and the correct spelling of Viget 🤪.
Example 1
The app asked: “Describe a time when you had to deal with a high-pressure situation. How did you handle it?”
My answer
“So recently, I worked with a cybersecurity startup — a funded startup. I worked directly with the five-person leadership team, most closely with the CEO and technical co-founder. It was a team of two: me and a designer. I was leading.
And the pressure was, we didn’t totally understand the ask; they couldn’t articulate it too well. Neither of us knew anything about cybersecurity. This was my first exposure to it. Some of the team members, some of the leadership team was skeptical we could ramp up on cybersecurity. They were skeptical that we could work with them as technical co-founders.
And so I handled it, first, knowing that I have a depth of experience that I can lean on to be confident that I’ll get through it. So it didn’t feel high-pressure in the moment, though it honestly was.
And so I just followed my process. I did really deep, thorough discovery; talked to every one of their team members one-on-one; did workshops together to understand the business, their problems, ramp up on the domain; did a ton of independent domain research, competitive analysis. And through that, leaned on my skillset and superpower of sense-making to not only kind of say back to them what I heard, but really was able to articulate and synthesize things that they couldn’t — that they still said in fuzzy ways or thought of in fuzzy ways.
So lots of aspects of the industry turned out to be really fuzzy. I was able to give a clearer concept model. I was able to clarify what their real goals are, which was to really reach a new customer segment. And I was able to more clearly articulate the problems they were hoping to solve and give them much more valuable and unique solutions than the kind of surface-level ones that they had initially thought of.
So as a result, we earned their trust. They implemented one of the first of those feature suggestions recently and used it as part of a major partnership announcement with Google.”
The app’s feedback
“That’s an incredibly detailed and robust approach to handling a high-pressure situation, especially in an unfamiliar domain. Your methodical and comprehensive strategy—deep discovery, individual conversations, workshops, and domain research—demonstrates your ability to manage complex challenges effectively. Employing your sensemaking skills to clarify and synthesize information not only helped you understand the client's needs but also contributed to building trust and delivering value.
Your experience underscores the importance of process, research, and communication skills in navigating high-pressure environments. Moreover, the outcome with the leadership team and the implementation of your solutions, leading up to a significant partnership announcement, is a testament to your successful handling of the situation.
Keep this story in your interview toolkit; it’s a solid example of managing pressure and producing results.”
Example 2
The app asked: “Can you describe a situation where you had to make a decision without all the information you needed? Take a moment to consider your answer, and whenever you're ready, go ahead and share your experience with me.”
My answer
“This is actually from a personal project.
So I am a sing-along impresario — I host sing-alongs — and I’ve created my own songbook. And I received feedback earlier this year from three different women who attended that the song keys were not good for them — that many of the songs, it was either too low or too high. Now, I don’t know what the best key is to change them into, but my audience is about 80% women, and so I wanted to just make a better experience for them.
And so what I decided to do is take a number of songs and just put them back into their original key. And I asked the people who gave feedback if they could listen to a few of the original songs, sing along with it, and tell me if the original key worked for them. And they gave me some feedback on that, and it seemed like a good enough option.
And so I went ahead and updated almost all of the songs that had female singers, even though I’m sure the new keys aren’t perfect either. But I just wanted to make the experience broadly better for them, even though I didn’t have all the information.”
The app’s feedback
“What a unique and interesting scenario! Your example perfectly illustrates your willingness to take action to improve your service, even when you don’t have all the information. It’s commendable that you listened to your audience’s feedback and took proactive steps to make the experience more enjoyable for them. By reverting the songs to their original keys and seeking further input, you showed that you value user experience and are adaptable.
One suggestion for improvement could be to discuss how you might measure the success of these changes. For example, you could collect more feedback after the changes or observe participation levels to ensure the modifications had the desired effect.”
Example 3
The app asked: “Can you tell me about a time when you took the initiative to solve a problem before it became a major issue? Reflect on your past experiences, and when you’re ready, I'm here to listen to your story.”
My answer
“Sure. So it was introducing, creating a QA process and structure at Viget for the first time.
So QA was one of the first product management things that I picked up. I was very adept at it and it just made sense to me. It took me years for other things, but I did that from the start. And when I started working at Viget, it was interesting to me that I kept hearing this chatter from devs, front-end devs, “We really need to hire some QA folks and a dedicated role.” And that seemed weird to me because I did QA on every one of my projects. I did it very well. So I wanted to dig into that.
So proactively, I basically did a QA initiative and I met with some of those developers, front end devs, who I had heard that feedback from, met with some of the project and product managers. And my assessment was, no, we didn’t need a dedicated role. The problem was sometimes QA isn’t happening or when it is happening, it’s not being done well. And the reason for that is lack of systems, structure, process, and training. And just people didn’t know about QA or know how to do it that well.
And so from there, I created our first QA process. So I wrote a doc that explained how we think about QA; what are the different buckets of QA — functional, visual, accessibility; which of those we do by default on projects, some performance testing we wouldn’t necessarily do; laid out those expectations. We used that doc internally and we would give it to clients on subsequent projects, so they understood what we would be doing QA-wise. I also created QA spreadsheet templates for a number of different scenarios: browser testing, complex multi-user testing, things like that. And then I did training for the PMs to help ramp them up on that.
And that was about 10 years ago. And that QA process is still the foundation of the current one today.”
The app's feedback
“Taking the initiative to assess and address the QA process at Viget showcases your foresight and problem-solving skills. You didn’t just take the conversation about needing a dedicated QA role at face value; you investigated the issue, gathered insights, and determined that the root cause was a lack of process, structure, and training. Your solution was comprehensive, involving the creation of documentation, templates, and training that not only met the immediate needs but also stood the test of time.
This example speaks volumes about your ability to identify gaps, take ownership, and implement solutions that have lasting impact. Your story also emphasizes the importance of understanding the nuances of your team’s challenges and being proactive in creating efficiencies.
Your approach to solving this problem before it escalated further is exemplary, and it’s the kind of initiative that employers look for in a leader. Be sure to highlight this proactive nature and your capacity to build sustainable systems when discussing your experiences in interviews.”
Why an AI job interview (practice) was so different
I wasn't shouting into the void
One of the most demoralizing aspects of a job search is the constant feeling of shouting into the void. You form a parasocial fantasy relationship with each job listing, imagining that since you applied — and maybe even got a tangential introduction — there’s obviously a real connection this time. Then applications go unanswered. You get ghosted after a final-round interview. Post-interview rejections convey little decision information. You realize the grinding reality of your situation.
Even worse, interviews with actual people often feel like shouting into the void.
Interviewers can’t or don’t engage with your answers (for legitimate reasons). You answer a question; at most they give a short, non-committal acknowledgement that you did, indeed, answer the question; then they ask the next question. You might as well be talking to a — well, I’d say “machine” if that weren’t the whole point of this article.
As soon as the AI app gave its first feedback, I almost cried. This was the first time during skill or behavioral interviews that I felt like the other person party actually engaged with my answers — that I got any feedback or engagement at all.
My experience felt valued — and validated
As the app gave feedback, I realized this was only the second time in my job search that my experience seemed truly valued and validated.1
There’s a weird dynamic in many job interviews where the employer is ostensibly already interested and impressed, but they rarely convey that out loud.
Instead, I found myself having to make the case that they should be interested in me, seemingly from scratch. I would think, “Wait, didn’t I get an interview because you’re already interested and impressed? If you’re not, why didn’t you just reject me? Why aren't we digging into the parts of my resume or cover letter that resonated the most — have you even read my resume?” It’s somewhere between gaslighting and playing hard-to-get. (Of course sometimes it’s because … they aren’t interested or impressed 🙁.)
In contrast, the AI coach was actually impressed with my experience!
Consider these feedback excerpts:
That’s an incredibly detailed and robust approach to handling a high-pressure situation, especially in an unfamiliar domain.
This example speaks volumes about your ability to identify gaps, take ownership, and implement solutions that have lasting impact. ... Your approach to solving this problem before it escalated further is exemplary, and it’s the kind of initiative that employers look for in a leader.
It was like: Ohmygodthankyou, finally!!! I generally don’t have imposter syndrome and am quite confident in my past work. (Don’t worry — I have plenty of other self-criticism to deal with.) But this AI feedback made me realize how much I had started doubting myself in the job search. It was so reassuring and needed.
My interview answers felt validated
It wasn’t just my experience that felt validated — I was also getting validation about my interview answers for the first time.
Consider these feedback excerpts:
Keep this story in your interview toolkit; it’s a solid example of managing pressure and producing results.
What a unique and interesting scenario!
The AI coach was impressed with my answers! Not the experiences and skills they reflect, but their effectiveness as job interview stories.
I almost cried a second time when the app called my sing-along answer “unique and interesting.” I’d had that story in my back pocket, but had been nervous about whether to use a non-work — and non-digital-product — example. Now I feel confident to use it in a real interview.
Again, contrast that with the typically stone-faced human interview responses to your skill and behavioral stories. You have no idea if your answer resonated, was what they were looking for, was too vague, too specific, too long, etc.
People are terrible at synthesizing interview answers
The last big difference between an AI job interview and a human interviewer: People are comparatively a joke at synthesizing show-don’t-tell interview answers.
Consider these feedback excerpts from the AI app:
Your example perfectly illustrates your willingness to take action to improve your service, even when you don't have all the information. ... By reverting the songs to their original keys and seeking further input, you showed that you value user experience and are adaptable.
This example speaks volumes about your ability to identify gaps, take ownership, and implement solutions that have lasting impact. Your story also emphasizes the importance of understanding the nuances of your team's challenges and being proactive in creating efficiencies.
There is no way a human interviewer can synthesize a candidate’s answers so thoroughly and insightfully, in written feedback let alone in-real time. Doubly so if a regular employee is conducting the interview, rather than People Team members or other ostensible job interview pros. (But even the latter are likely miles away from the AI app’s synthesis abilities.)
If you’ve ever conducted a job interview, look at the above AI synthesis and honestly evaluate your synthesis in comparison. I have, and my honest answer is: ell oh ell.
AI and the future of story-based job interviews
Based on this experience, if I have to do story-based interviews (i.e., answer skill and behavioral questions), I would 100 percent prefer to do it with an AI rather than a person. (I think people should absolutely still facilitate other kinds of interviews like portfolio shares and exercises / simulations. And people, not AI, should make the decisions throughout.)
Some hypotheses for why an AI interviewer could be better for story-based interviews:
Less demoralizing
An AI interviewer could engage, even provide feedback, without giving inappropriate false hope. (Assuming the app sets appropriate expectations for the candidate.)
Compounded across a candidate’s interviews, this would be a powerful antidote to the demoralizing, shouting-into-the-void aspects of a job search. Save demoralizing for the actual rejection: there’s no need to make the rest of the process feel like an impersonal, ongoing rejection-in-the-making.
Reduce pressure
Story-based job interviews can be uniquely stressful for candidates, in ways that I believe are not an effective proxy for simulating on-the-job pressure.
For one thing, in a job interview you simply aren’t fluent with the material the way you would be when giving a work presentation. After all, nobody perpetually maintains and practices a body of STAR-formatted interview answers about their work experience.
For another, job interviews often carry layers of fraught personal dynamics that aren’t present in job situations. The stakes are existential. You — not your work — are being judged. Your deepest self-criticism and regret are always just below the surface, as you’re forced to reflect on your past and constantly talk about yourself.
I suspect an AI interviewer would mitigate some of those fraught dynamics, even though the existential stakes are ultimately the same.
10x better synthesis
The feedback excerpts above speak for themselves. If story-based interviews are supposed to assess how well a candidate’s stories demonstrate skills and behaviors, an AI app can articulate that assessment so much better and faster than a person. This would both improve outcomes — employers have much better information to base decisions on — and save employers a ton of time.
Bias mitigation
AI interviews could enable blind auditions at scale. And the 10x Better Synthesis factor would likely identify promising experience that human interviewers might overlook.
Wrapping Up
All of this assumes story-based job interviews are actually useful, predictive, necessary, etc. in the first place. Hot take: I don’t believe they are — but that’s a topic for another post!
In the meantime, if you’re applying to jobs I highly recommend using Nadia or another AI app to practice your job interviews.
The first time: A recruiter emailed out of the blue, after finding my LinkedIn profile and seeing potential for a match. While the opportunity didn’t pan out, her interest and validation meant so much amid the aforementioned void. Forever grateful, Kelly P.!