Cheap agents just got here. Test them on your own work before you ship.

Two things happened in the same week, and together they change how you should pick a model. Anthropic shipped Sonnet 5, a cheaper model that can run agents on its own. And Arena, the leaderboard people trust, hit $100 million in revenue selling one thing: proof of which model is actually good at what. Let me catch you up on what that means for your roadmap.

The price of running an agent just dropped

Anthropic pitched Sonnet 5 as its most agentic Sonnet yet, able to plan, use browsers and terminals, and work on its own at a level that used to need a bigger, pricier model. It launched at $2 per million input tokens and $10 per million output, cheaper than Opus, GPT-5.5, and Gemini 3.1 Pro.

The numbers back part of the story. On agentic coding, Sonnet 5 scored 63.2% against Opus 4.8's 69.2%, and on a knowledge-work benchmark it slightly beat Opus. Anthropic's own line is honest about the tradeoff: Opus is still the pick for the hardest judgment calls, but Sonnet 5 gives you higher quality at a lower price than you had before.

So the differentiator moved. It is not who does agentic work. It is who does it cheaply and reliably without someone watching. That is a real shift for anyone budgeting a feature.

From chatbot helper to work you hand off

Ethan Mollick calls this the twilight of the chatbots. The old way was a co-pilot: you ask, you check, you ask again. The new way is handing a whole job to an agent and managing it like a report.

The capability gains are steep. Mollick cites Epoch finding Opus 4.7 worked 14 hours on its own to build software that would take a human 2 to 17 weeks, for $251 in tokens. A joint OpenAI study found a quarter of its workers run at least four agents at once every week, and legal and HR picked up agents nearly as fast as coders did.

Here is the part that should shape your hiring and training: what predicted success with agents was not the person's job title. It was their domain expertise. The deeper someone knew the work, the more useful output they pulled from each prompt.

The benchmark that gets you fired

The catch with cheap agents is that public benchmark scores lie to you about your work. Mollick notes the open-weights models often do not perform as well as their benchmarks say. And Anthropic skipped the numbers on the safety claims that matter, giving only a general "lower rates" of hallucination instead of data.

This is why Arena turned into a business. It started as a Berkeley research project and now sells deep-dive analytics to labs and enterprises off over 10 million user evaluations. Its ARR went from $30 million in January to $100 million eight months after launching the paid service. People pay for someone to answer "which model is good at my task," because the leaderboard alone will not.

A generic score tells you a model is smart in general. It does not tell you it can update your Salesforce tiers end to end without stalling halfway, which is the actual job.

Build the harness once, run it every release

Claire Vo did the version of this you can copy. Tired of one-off vibe checks, she built a repeatable eval harness in under 45 minutes and ran Sonnet 5 blind against four other frontier models across PRD quality, prototype generation, agentic tasks, and agent personality. Sixty-four generations, scored the same way every time.

Her scoring mix is worth stealing: 70% human gut-feel, 30% LLM-as-judge, because she trusts neither one alone. The payoff was a model-by-task recommendation. One model for PRDs, another for complex prototypes, another for daily chatting. Not one winner. The right tool per job.

That is the move for your team. Not a slide about which model is best. A small, repeatable test built from your own past sessions, rerun on every release.

The deep cut

Anthropic is not really selling a model anymore. Claude Science, its new flagship for scientists, runs the same Claude models everyone already has, with no special access. What it sells is the workflow: 60-plus databases, sub-agents, and a fact-checker step wired into one place. The bet is on owning how the work gets done, the way Claude Code owns software.

So model choice is becoming a commodity decision, and the value moves up to the workflow you wrap around it. For you that cuts two ways. Do not marry a feature to one model, because a cheaper one that scores the same on your eval will show up in months. And spend your real design effort on the harness, the tools, and the guardrails, because that is the part that lasts when the model underneath swaps out.

Three questions for your team

1. What are the five tasks we would actually hand an agent, and do we have a repeatable eval for them, or just vibes and a public leaderboard?

2. Which features are hard-wired to one model right now, and how fast could we swap in a cheaper one that passes the same test?

3. Where is our design effort going: into picking a model, or into the harness and guardrails that keep paying off when the model changes?