AI Found One Real Bug. It Also Made Up Three That Never Happened.

If you run research or own product decisions, you have heard the pitch: AI can run your usability tests, fake your participants, and cut your sample sizes. Some of that is real. A lot of it is sales talk dressed up as science. MeasuringU spent the last few weeks pulling the curtain back, and the picture is sharper than the hype.

Let me catch you up on what they found, and what you should do with it before your next review.

AI is a junior researcher, not a senior one

Here is the headline test. MeasuringU had four human researchers watch a six-minute video of someone booking a dinner reservation. Then they ran the same video through ChatGPT and Gemini. The AIs flagged 11 problems no human caught. Sounds great, until you check the work.

Of those 11, only one was a real bug. Seven were false alarms. Three were flat-out made up. The single real find was good: Gemini caught the participant using Ctrl-F to search for "sushi" on a page that didn't list it, a sign the filter design failed. But to get that one gem, someone had to review ten other claims that were either wrong or irrelevant.

The math here matters. Roughly nine out of ten AI-only problems needed a human to correct or toss them. The author's own line lands it: AI "can add value to this type of UX research, but more as junior researchers whose actions and conclusions require expert human oversight rather than as trusted experts."

The hallucinations you can't see coming

False alarms are annoying but honest. The AI saw something real and read it wrong. ChatGPT got fixated on the word "sushi," decided a restaurant labeled "seafood" didn't count, and declared the task a failure in three of four runs. A human shrugged and said the place serves sushi, task done.

Hallucinations are the real trap. The AI reported things that never happened. Gemini claimed the participant picked the highest price tier. She picked mid-range. Gemini said she set the time to 5:10. She set it to 5:00. ChatGPT said she never reached the reservation form. She did.

Here is the part that should change how you staff this: you cannot tell a hallucination from a real find without watching the video yourself. The fabricated claims read just as confident as the true ones. One useful trick from the study is to run each AI pass several times and look for what holds across runs, since two of the three hallucinations came from a single shaky Gemini run. But that is a filter, not a fix. Human review is not optional.

Not all fake users are faking the same thing

Then there is the synthetic users pitch: skip real people, generate their answers with AI. MeasuringU's move here is useful because it stops the word "synthetic user" from meaning everything and nothing. They laid out five types, sorted by how grounded they are in real human data.

At the weak end is the AI proto persona, built from a one-line prompt like "you are a world-class Python programmer." That is guessing. Then demographic-based, then persona-based, both still loose. Stronger is research-grounded, which pulls from real interviews and analytics. Strongest is the digital twin, modeled on one real person's data, though the authors are honest that its accuracy in the field is "still an open research question."

The point for you is simple. When a vendor says "synthetic users," ask which type. A persona built from a prompt and a twin built from your own customer data are not close cousins. Treating them as the same thing is how teams get burned.

The 30-participant rule was never really a rule

While AI gets the headlines, MeasuringU also went after a quieter waste of money: the belief that you always need 30 participants. You have heard it from a skeptical stakeholder or a peer reviewer. It feels like a law. It isn't.

The number has real roots. Around n = 30, the t-distribution lines up with the z-distribution, and a wide mix of data shapes settle into a normal sampling distribution. But that is where the math stops being a law and starts being a habit. As the authors put it, the rule "calcified into a general-purpose sample size rule" when it was never meant to be one.

For common UX data, the floor is lower. Bootstrap tests showed SEQ and SUS scores reach a normal sampling distribution by n = 10, completion rates and times by n = 20 to 30. The t-distribution was literally built for small samples by a Guinness brewer who couldn't take big ones. So insisting on 30 every time is "wasteful at best and infeasible at worst." Pick your sample from the question you're answering, not a number you half-remember from school.

The deep cut

The thread running through all of this is the same: AI makes it cheaper to produce research-shaped output, and that is exactly the problem. A confident AI summary, a synthetic persona, a tidy n = 30 study can all look finished while being wrong. The work that's getting cheaper is generating findings. The work that isn't is verifying them.

So do not let AI cut the wrong line item. Use it to widen coverage, run more sessions, draft more notes. But keep, or grow, the human time spent checking. The genuine Ctrl-F find is real value. It only counted because a researcher watched the tape and could tell it apart from the made-up price tier. Budget for that review, or you are buying noise that reads like signal.

Three questions for your team

• When a vendor pitches "synthetic users," which of the five types are they actually selling, and is it grounded in our data or just a prompt?
• For our next study, what's the real sample size the analysis needs, and are we defaulting to 30 out of habit?
• If we use AI to review session videos, who watches the tape to separate the real finds from the false alarms and hallucinations, and is that time in the budget?