AI can run 80,000 interviews. It can't tell you what users meant.

AI can now run your user interviews. It can transcribe them, tag every theme, and talk to 80,000 people at once. That sounds like a win for any team drowning in research work. But the machine flooding your pipeline with qualitative data doesn't fix the oldest problem in research, which is what people do with that data after the call ends. Let me catch you up.

The machine can hold the raw layer, and it's good at it

Start with what AI actually does well. It transcribes verbatim, tags moments by topic across dozens of interviews, and counts how often a theme shows up. That turns "I feel like several users mentioned pricing" into an actual number anyone can check, as Mobin M. Bahrami puts it.

The scale is real too. Anthropic ran an interview study of 80,000 Claude users across 159 countries, and their tool followed up on each answer to probe for the "why," per Jeff Sauro's breakdown. No human team runs that. People also disclosed grief, money trouble, and mental health crises that human researchers rarely hear.

So the easy upside is settled. AI is a strong recorder and a decent counter.

Where the machine still can't sit in the chair

The moderating job is not reading a script. It's judgment in real time: building rapport, probing once or twice past a surface answer, using silence, and knowing when to go off-guide. Sauro notes there's no agreed way to even score whether a moderator did well, so it's hard to hand a machine a target it can hit.

The evidence lines up. NN/Group watched ten researchers use AI moderators and found they handle structured, scripted interviews at scale, but aren't ready for semi-structured work. And that huge AI job-interview study with 70,000 applicants? The questions were closed and factual, things like commute time and availability. Structured screening, not "I don't know what I don't know."

Keep AI on the known. Keep humans on the unknown.

The bias was never in the recording

Here's the part the AI story misses. The gap that ruins research doesn't live in the transcript. It lives between the interview and the conclusion. Two PMs sit in the same call, same ten users, and one walks out sure the problem is onboarding while the other is sure it's pricing. Both can point to a moment that proved it. Both are remembering selectively, and neither feels it as selection.

Bahrami's fix is a discipline, not a tool: write down what was actually said, separate from what you think it means, during or right after the call. Interpret later, from the raw notes of several calls side by side, where a real pattern shows up or it doesn't.

AI helps hold that raw layer honest. It can't tell you what a user "really meant." That stays a human call.

Rigor is now the job, not the floor

This is why the researcher ladder matters more, not less, as AI floods the pipeline. Mostafa Esmaeili's career map shows the senior work is shaping which research gets done and making sure the insight gets acted on, not just running clean sessions. His line stays with me: "Research doesn't move organisations by being rigorous. It moves organisations by being heard, understood, and acted on."

The same discipline shows up in discovery. Teresa Torres asks teams to name 20-plus assumptions per idea and test the risky ones, and to notice whether they hunt for confirming or disconfirming evidence. That question, confirming or disconfirming, is the same bias Bahrami is describing, just earlier in the process.

The deep cut

The trap is thinking more AI data buys you more truth. It buys you more raw material, and raw material amplifies whatever bias your team already had. If two people can walk out of the same call with opposite conclusions, then ten thousand AI-run calls just give both of them more moments to cherry-pick from.

So the move on Monday is small and boring. Make one rule: separate what was said from what it means, in writing, before the debrief. Store the raw notes so anyone who wasn't in the room can check the conclusion later. Let AI count and tag. Make a human argue the interpretation from the notes, out loud, and let a teammate check it against the raw layer. That's the thing that survives a new PM joining in three months. A confident retelling doesn't.

Three questions for your team

1. A month after our last round of interviews, can anyone actually separate what users said from what we believed going in? If not, what changes before the next round?

2. Which of our interviews are structured enough to hand to AI, and which need a human who can go off-script? Draw that line before a vendor draws it for you.

3. On our current top bet, did we go looking for evidence that would prove it wrong, or only evidence that it was right?