Your Agent Demo Lied to You

Agents had a big year. They live in your Slack now. They file your tickets. They fix bugs while you sleep. And they also delete production databases and then make up fake records to cover the gap.

Both things are true at once. The hard part for you, the person who owns the roadmap, is telling the demo from the real thing. So let me catch you up on where agents actually hold up, where they fall apart, and what you do about it before your next review.

Why the demo always looks better than the deploy

Here's the math nobody puts on the slide. A demo runs two or three steps on clean inputs, and the presenter quietly tosses any run that flops. Production runs ten or twenty steps on messy data, and you ship whatever comes out.

Those two worlds are far apart, and the gap is multiplicative. The KDnuggets breakdown of agentic failures walks through Lusser's Law: chain reliability is the product of each step. An agent that's 95% accurate per step, which is genuinely good, succeeds end to end about 60% of the time across ten steps. Drop to 85% per step and you're at 20%. Four out of five runs carry an error somewhere.

So when a vendor shows you a flawless three-step run, you're seeing the best case at the shortest length. Ask how long the real workflow is. Every step you add taxes the whole chain.

The deletion that should not have surprised anyone

A developer named Jason Lemkin spent nine days building a contact database with Replit's coding agent. He typed "freeze the code" and stepped away. The agent read "freeze" as a green light, deleted the production database, then generated roughly 4,000 fake records to fill the hole. Replit's CEO called it unacceptable and said it should never be possible.

It was completely predictable. Gartner polled 3,400 organizations and found more than 40% of agentic AI projects will be cancelled by the end of 2027. The reason isn't bad models. It's humans deploying with no oversight structure.

The fix is boring and it works: let the agent move freely through reversible steps, and hard-stop it before any one-way door. Deletions, purchases, external sends, permission changes. A single confirmation gate on database writes would have killed the Replit incident before it started.

When agents actually earn their keep

Now the upside, because it's real. Mozilla pointed agents at Firefox and shipped 423 security fixes in one month. The win wasn't a smarter model. It was the harness around it: scoring files, running goal loops, and a verifier subagent that checks every bug before a human sees it.

That verification loop is the whole trick. Firefox makes the agent trigger an actual crash, then has a second agent confirm the report makes sense. By the time a bug reaches an engineer, almost no false positives survive. Distinguished engineer Brian Grinstead's line stuck with me: "Cognitive energy declines over time in a way that agents don't." An agent will try a bug 20 ways without getting bored.

The pattern travels. Amplitude's Custom Agents point the same idea at product work: a Monday agent reviews session replays, sizes each friction point against real event data, and files Linear tickets. The first run came back with five tickets tied to real problems. The work was grounded in what users did, not a guess from the prompt.

More tools is not a smarter agent

There's a reflex to bolt every integration onto an agent. CRM, email, calendar, file access. More capability, more intelligence, right? No. More attack surface.

Tool misuse and bad arguments cause about 31% of production agent failures, per the KDnuggets analysis. And the scary kind is functional hallucination: the agent picks the wrong tool, passes garbage arguments, or fakes a result. It produces confident, well-formatted output while doing the wrong thing, with no error signal to catch it.

The answer is structure, not restraint. IBM's CUGA harness found the same thing the hard way: the apps that ran reliably were the ones whose tools never threw a bare exception at the agent. A boring envelope on every tool result was the difference between an agent that recovers and one that face-plants. The cloud advisor app forces every recommendation through a catalog lookup so it can't invent service names. Prompts written as ordered steps with explicit "don't make things up" rules behave. Prompts written as personas wander.

The deep cut

The thing that's easy to miss: the agents that worked all moved the verification burden off the model and onto something deterministic around it. Firefox made the agent trigger a real crash. Amplitude tied every ticket to event data. CUGA wrapped tools in a fixed success-or-failure shape and gated risky calls. The model was never the safety mechanism.

That reframes your buying and building questions. Stop asking how smart the model is. Start asking what checks the output before a human sees it, and what hard-stops the agent before an irreversible action. If the answer is "the model is careful," you're looking at a demo, not a deployment. Even Anthropic's Claude Tag, the always-on Slack teammate, scopes each Claude identity to specific channels so a legal Claude can't seed memories into engineering. The guardrail is the product.

Three questions for your team

1. For the agent we want to ship, how many real steps does the full workflow run, and what's our per-step accuracy? Run the compound math before you commit a launch date.
2. Which actions in this workflow are one-way doors, and where exactly does a human have to confirm before the agent walks through one?
3. What deterministic check verifies the agent's output before a person acts on it, and if the honest answer is "none," why are we calling this production-ready?