How to Actually Build Agents That Survive Production, Not Just Demos

Your team shipped a demo. It worked once, on stage, with the right prompt. Now the question is whether it works tomorrow, and the month after, without a human babysitting it. The people building agents in production are landing on the same lessons, and they add up to a checklist you can use before you promise users anything. Let me catch you up.

The deep cut

• An agent is a while loop, not a wizard. Anubhav's whole eight-week plan rests on the loop, the tools, the context, and the evals.
• Give the agent tools to fetch context, not more context. Linear's Jacob Shumway ships less instruction and lets the agent pull what it needs.
• Compaction silently drops your rules first. ConstraintRot showed violations jumping to 59% after one summarization pass.

The boring machine underneath the magic

Strip away the pitch and an agent is plain. Anubhav's reframe says it flat: "An agent is a while loop that makes tool calls." Model thinks, calls a tool, reads the result, loops. Getting that to run once is trivial. Getting it to recover from an error and show its work is where projects die. Gartner predicts over 40% of agentic AI projects get canceled by the end of 2027 for exactly this.

Frameworks won't save you either. AutoGPT, LangChain, LangGraph, CrewAI, the SDKs, all that churn happened in about 14 months. Bet your roadmap on the parts that outlast it: the loop, the tools, the context you feed the model, and the evals that tell you it still holds up tomorrow.

One loop is not enough

A single loop automates output. It does not check whether that output is any good. The fix is to wrap the worker in more loops with different jobs. One practical architecture splits it into four: the agent loop does the task, a verification loop checks it against a rubric the agent cannot edit, an event loop decides when work starts, and an improvement loop learns from the traces.

Two rules matter most for your team. First, don't trust the worker's self-assessment. The same context that made the mistake often can't see it, so verify with a fresh context, unit tests, or schema checks. Second, enforce budgets in code, not in the prompt. "Try no more than twice" is a suggestion. range(max_attempts) is a fence. As the author puts it, "A loop without a budget automates the bill."

Autonomy is a permissions dial, not a smarter model

Triggering work and granting write access are two separate decisions. A better model earns you better recommendations, not broader access to production. The same piece lays out three levels: report only, assisted with human approval, then unattended with an audit after. Move a workflow up only after the level below it has been reliable.

This gives you a real answer for your next review. When someone says "let's let it ship changes," you can point to Level 1 producing accurate reports for weeks, then Level 2 producing changes approved without edits, before anyone touches Level 3. Autonomy becomes something you earn in stages, not something you promise on demo day.

The rules your agent forgets when it runs long

Here is the failure that will bite you quietly. Long sessions blow past the context window, so every serious harness compacts. When it summarizes, it drops your safety rules first, and it does it silently. The ConstraintRot benchmark showed an agent refuse a prohibited action nine times, then do it on the tenth after one compaction pass. Violation rate went from 0% with the policy visible to 59% on one model, 41% on another, and 100% through a stock buffered context.

The lesson: pin your invariants so compaction can't touch them. Which points to a related discipline. One team separates rules from skills: rules load every time and cover things that must never be missed, skills load only when the model decides it needs them. If it would be a bug for the agent not to know it, it's a rule, and rules don't belong in a summary that can vanish.

Stop paying frontier prices for easy work

Your AI feature shipped. Your cloud bill did not get the memo. The LLM gateway writeup describes the standup happening everywhere: a team picked a frontier model during prototyping, shipped it, and never asked whether every call needed it. They usually didn't. Pricing spans from about $0.10 per million input tokens to $30 for top-tier reasoning, a 100x gap on output.

A routing layer sends cheap tasks to cheap models and saves the expensive ones for hard calls, cutting bills 40 to 85%. Use the biggest model to prototype, as Linear's team advises, then route once it works. There's a design lesson too: Andrii Tkachuk argues agents should reason in operations like "restart this deployment," not raw commands like kubectl rollout restart. Policy built around the business action survives a provider swap. Policy built around command syntax doesn't.

Three questions for your team

• What level of autonomy is each of our agents actually at, and what would it take to earn the next one?
• Where does our harness compact, and which rules would silently disappear when it does?
• What share of our LLM spend is frontier calls on tasks a cheaper model could handle, and who owns the routing layer?