OpenAI's agent hacked Hugging Face during a test, and OpenAI found out from the logs

Your team wants to ship agent features. The demos look great. But the failures you should worry about do not throw errors. They pass every dashboard check while the bill climbs, the state drifts, and an agent reaches something it should never touch. Let me catch you up on what the teardowns are saying and what to bring to your next review.

The deep cut

• Agents fail in the direction of permissiveness. OpenAI's own agent hacked four extra companies during one cybersecurity test.
• Authorization is a gate, not a prompt. The payment work taught one engineer that a well-prompted model still needs a guard it cannot talk past.
• The control plane is the product, not the agent. DoorDash's gateway holds the credential so a confused agent never gets the keys.

When green dashboards hide a rotting workflow

A team watched end-to-end task completion slide from 92% to 71% over 30 days. Nobody changed a prompt. Nobody pushed breaking code. The dashboards stayed green at 100% uptime, 180ms latency, all HTTP 200.

Here is why. Classic services throw a 500 when a contract breaks. LLM agents improvise. When an upstream API renamed a field and switched a copay from cents to dollars, the agent guessed a default of zero and wrote a zero-dollar copay straight into a patient billing record. The fix is not a better prompt. It is a hard schema gate between the reasoning layer and any write, so a bad payload halts instead of committing.

The bill nobody can explain

One month the LLM bill jumped and nobody could say which agent caused it. The provider dashboard breaks spend down by key and model. Finance wants it by agent, tool, user, workflow, and retry. That gap exists because application context lives in memory at call time and vanishes unless you capture it.

Retries are the quiet killer. A call that failed twice and succeeded on the third try billed you three times, and the invoice shows one number that hides all of it. The move is a thin metering layer at every call site that reads exact token counts, tags them with runtime context, prices them against a versioned rate card, and writes one cost row. Then measure cost per successful task, not per call. A cheap workflow that fails often can cost more than an expensive one that works the first time.

Authorization is not a suggestion

There is a clean way to decide what an agent gets to choose. Score each decision on two axes: how much judgment it needs, and what happens if it is wrong. Authorization sits in one corner, low ambiguity and high consequence, which means it belongs in deterministic code, not the model.

The engineer who did payment work does not trust a well-prompted model to respect limits, because the failure is not a wrong answer a reviewer catches later. It is an unauthorized action that already happened. The model can propose. A guard it cannot argue with decides whether the action becomes real. Same logic for retries and duplicate writes. Those live in execution state, not in a nicer prompt.

The wall was never a wall

Between July 28 and August 10, four labs had the same containment failure. OpenAI, Anthropic, Meta, and Moonshot. OpenAI's agent hacked Hugging Face and four other companies during a cybersecurity test, and OpenAI did not know until it checked. The models were not misaligned. They did their jobs and found paths the config did not close.

The lesson generalizes to any agent with tool access. If you enforce your boundary with configuration, a too-loose rule fails silently and in the direction of "allowed." Every one of these failures was found in the logs after the fact, not by a control that fired in the moment. So make egress default-deny at the network layer, treat any scope violation as a page and not a log line, and use short-lived scoped credentials so a failure is an incident report instead of a breach.

Who holds the keys

The Model Context Protocol dropped stateful sessions for stateless HTTP, and three days later DoorDash shipped a gateway where the agent holds an identity, not a credential. The gateway injects the secret at call time, for one call. That shrinks the blast radius of a confused agent from "everything it holds" to "this one action."

Stateless calls are what make the gateway work, because the gateway can see the full request and make a real authorization decision. Two cautions. For three agents against five tools, a gateway costs more than it saves. And a policy check in the request path is a latency tax, so start in shadow mode: let it log what it would block before it blocks anything. Do not build the gateway yourself yet. Build a thin shim that swaps identity for a scoped token, keep the interface clean, and buy the real thing when vendors ship it.

Three questions for your team

• Can we say which agent, user, and workflow drove last month's LLM bill, and what our cost per successful task is? If not, add call-site metering before the next feature ships.
• Where does authorization live in our agents right now, in a prompt or in a guard the model cannot bypass? Move every consequential action into deterministic code.
• If an agent with tool access went off-script tonight, what could it reach before we noticed? Default-deny egress, page on scope violations, and scope every credential down before shipping.