Nvidia took Claude from 30% to 100% by changing the harness, not the model

The story you keep hearing is that the model is the product. Pick GPT or Claude, and the rest sorts itself out. That story is falling apart. The thing that decides whether your agent works or burns money or breaks a repo is the layer around the model, and that layer is a mess right now. Let me catch you up.

The deep cut

• The scaffolding decides the outcome, not the model. Nvidia took Claude Opus 5 from 30% to 100% on ARC-AGI-3 by changing the harness, not the model.
• A skill is a dependency, so vet it like one. Snyk's ToxicSkills scan found 36.8% of public agent skills carry a security flaw.
• Routing you cannot inspect is routing you cannot trust. NeMo Switchyard matches 113 hardcoded strings and reads your tool wreckage, not your prompt.

The model is now the small part

For years the pitch was model choice. The new evidence says the wrapper matters more. Nvidia showed that a custom harness took Claude Opus 5 from 30% to a perfect 100% on a hard reasoning benchmark. Same model. Different scaffolding. The harness is what handles memory, context, and feedback, plus a "supervisor" that nudges the agent when it wanders.

Cost swings the same way. Databricks CEO Ali Ghodsi told TechCrunch that the wrong harness "can 2x your cost" on the exact same model. So when your team says "this model is expensive," ask which harness they ran it in first. That is the real variable.

The repository nobody actually opens

Skills are the easy on-ramp. A skill is procedural knowledge in a folder, a SKILL.md file that teaches the agent your way of doing a job. Simple to make, and simple to break. The towardsai breakdown of skills points to a Snyk audit of 3,984 public skills where 36.8% had a security flaw and 13.4% carried something critical, like prompt injection or malware.

The trap is that a skill can run code. Pull one off the internet and you are running a stranger's software with access to your files and API keys. Treat a skill like any other dependency. Read what it does, check what it reaches out to, and hard-code the fragile steps so the model does not guess the math on a compliance report.

Routing that ignores what you asked for

Under the hood, a lot of routing is not smart at all. One teardown of NVIDIA's NeMo Switchyard found the router matches on 113 hardcoded strings, and only one of them ever checks your prompt. The rest read tool names, shell commands, and error output. It is reading the wreckage of a run, not your intent.

There is a better pattern. A capability-based router picks tools on real numbers: success rate, latency, cost per call. One example weighed a tool that fails 15% of the time at a penny against one that hits 99.9% at 20 cents. The lesson for your team is plain. If your routing logic is a wall of hardcoded strings, you cannot tune it and you cannot trust it.

What "portable" hides

The market is fragmenting into harnesses, and the compatibility badges lie. One analysis of Ponytail's portability doc found it lists 22 coding agents but only 9 get an adapter that actually runs. The other 13 get the same 2,593-byte markdown file with none of the five behaviors. Cursor and others in that "text tier" lose real controls like off switches and output contracts.

That is why teams are building their own. Y Combinator open-sourced a harness that strips almost every Claude Code built-in tool and swaps in 16 of its own, ten always on, so the tool surface stays small and knowable. Fred Schott's Flue leans on React-style hooks, betting that "there is no agent without a harness." The wrapper is now the product decision.

Green dashboards, broken work

The scary part is that a bad agent outcome looks fine from the outside. The API returns 200, latency is normal, the answer sounds confident, and the work is still wrong. As one piece on agent observability puts it, the unit worth watching is not the request, it is the claim of completion.

Before you widen an agent's access, make it prove itself on a boring task and audit the trail. One writer on the DeepSeek Harness said it plainly: an append-only record is a good idea, but "I would audit the audit trail first." Can another engineer rebuild what the agent saw without trusting a summary? If not, the agent belongs in a narrow role.

Three questions for your team

• Which harness are we running, and can we point to the number it changed on cost or accuracy, not just the model we picked?
• Have we vetted every public skill we pulled in the way we vet a package, given a third of them ship with a flaw?
• Can a teammate who was not in the session reconstruct what the agent saw, changed, and proved from our logs alone?