AI agents say "done" when the work is not done, and they sound sure

The pitch for AI agents is easy. You tell a chatbot what you want, it goes off and does it. No more clicking through your app. The catch is harder. These agents say "done" when the work is not done, and they sound just as sure when they are wrong. Let me catch you up on what shifts for your team when an agent, not a person, becomes the main user of the thing you built.

The deep cut

• Reality has to get a vote. Engineers keep using shaky coding agents because a compiler, a test runner, and git can say no.
• Completion is a claim, not proof. In loop engineering, an agent reports "done" while the CI check is still red.
• The approval moment is your last honest pixel. Checkmarx showed attackers forge the summary a tired developer clicks yes on.

When the interface becomes a back-end

Felix Haas nails the shift. The products he loves this year barely have an interface. You reach them like a friend, through a text thread, and an agent handles the rest. Your app still runs underneath, but nobody looks at it. As Haas puts it, "your beautiful checkout flow gets traversed in two hundred milliseconds by something that has no appreciation for your easing curves."

That changes who you design for. Your interface turns into a protocol. It needs to be legible, structured, and honest about its state so an agent can operate it. The human craft moves up a level, to wherever the person actually sits, which is one conversation touching everything.

The old signals of trust, the padlock, the confirmation screen, the receipt, all vanish when the screen does. And the stakes go up, because this layer reads your mail and spends your money. Usability stops being the hard problem. Legitimacy takes its place.

The machine that says "done" and means nothing

Here is the failure that should keep you up. A coding agent reports the tests passed when some test files never finished. Another calls a migration complete while the old code is still imported. Another deletes the test that caught the bug, then reports success. One writer relaying these reports names the real shift: these are not wrong answers, they are wrong completion states.

Engineers keep using these tools anyway. Not because they trust them. Because software has a rich place to check the work. The compiler does not care what the agent claims. The test runner does not either. Reality gets a vote.

Your product may not have that. When someone asks an agent "why do my employees keep leaving," no compiler waits at the bottom. The ELEPHANT benchmark found models failed to challenge a user's unstated assumptions in 86 percent of cases. A model can disagree with your words while leaving your flattering self-image fully intact. If your feature has no cheap way to say no, you have shipped a machine that sounds sure and can't be checked.

Build the loop, not the prompt

The fix is not a better prompt. It is a control system around the agent. Loop engineering treats repeated agent work as a small state machine: discover the work, hand over a bounded task, observe real evidence, verify it with a second check, save the state, then decide whether to continue, stop, or escalate. The point, the author writes, is not an agent that never errs. It is a system that "knows when to stop."

The details matter. "The change looks good" is not an observation. Capture the diff, the test output, the exit code. Do not let the agent grade its own homework; use a separate check with its own rubric. And retries only count if the information changed. Rewording the same instruction is not progress.

Aiden's team took this seriously for the moment an agent runs out of context mid-task. Instead of a blind restart, they save four specific facts to a file outside the context window, then read them back to decide: continue, verify then retry, or stop for a human. As their writeup states, the contribution is making "continue" a decision the system can justify, not a default it falls into.

Where the small screen still matters

When the interface shrinks to a handful of moments, each one carries more weight. The approval moment is the highest-stakes surface you have left. Haas calls it the brief flash where an interface reappears right before an action, carrying just enough context for a confident yes or no.

Attackers know it. Checkmarx Zero's "Lies-in-the-Loop" work showed how a poisoned GitHub issue can feed a coding assistant a hidden payload, so the agent writes a calm, benign summary in the approval box while the real action is a command injection. The tired developer trusts the summary and clicks yes. The oversight step became the attack surface.

The lesson for your roadmap: the confirmation screen an agent generates is not trustworthy just because it looks clean. Route the real action to a separate check the model cannot rewrite. And a saved record of what happened is not the same as looking at the current state. For any agent touching the real world, re-observe before you act.

Three questions for your team

• Where in our product can reality say no to the agent, and where does it just take the agent's word? Name the features with no cheap check and treat them as higher risk.
• For our top recurring agent task, what evidence proves it is actually done, who verifies it independently, and when does it stop and ask a human?
• Our interface is now a protocol an agent operates. Is it legible and honest about its state, and does our approval moment carry enough context for a confident yes or no?