Your Agent Isn't Listening, and Nobody's Checking

You point an AI agent at your product and it ships a settings page in seconds. The spacing looks fine. The components are real. But the button is a shade of blue you never approved, and the form is centered when every other form you have is left-aligned. Nothing broke. Everything is a little off.

That gap is the whole story right now. Agents can generate UI fast, and the fast part is easy to love. The hard part is making sure the output is actually yours, and knowing when it drifts. Let me catch you up on the playbook teams are building to keep quality and control while the agents do more of the typing.

The stranger who rearranges your furniture

An agent doesn't read your mind. It pattern-matches your words. Type "design a dashboard" and you get the statistical average of every dashboard it has ever seen. The Nielsen Norman Group named the result the Frankenstein layout: every part is recognizable, the whole thing feels randomly assembled.

The fix people keep landing on is a briefing habit, not a clever prompt. Say who the screen is for, what real content goes on it, how it should feel, and what is off-limits. Miro's design team calls their version RTCF: Role, Task, Context, Format. The label matters less than answering each part every time instead of skipping to the task.

This matters more than it used to because of who is doing the prompting. One build-economy analysis found founders make up nearly half the people building with AI tools, and about four out of five have no technical background. Prompting is now the primary interface for people who never learned Figma.

The file that tells the agent your rules

Most teams building with agents already have a CLAUDE.md that explains how the codebase works. Far fewer have the design equivalent. So Rishi Sharma watched an agent redesign a settings page that did everything he asked and nothing he meant, because it had no idea the product used 4px spacing, that destructive actions are always red and always confirm, or that modals got dropped for anything longer than two fields.

A design.md carries that context. Spacing increments, color logic, the toggle pattern you standardized on, the components you deprecated. It is the same repeatable move the prompting guides push: pull the decisions that made a good screen work, and fold them into every prompt that follows.

Accessibility lives in the same blind spot. Contrast, focus order, label structure. An agent won't volunteer any of it unless you write the constraint down once, in the file, instead of remembering to mention it screen by screen.

Writing the rules is not the same as following them

Here is the part that trips teams up. You can write the cleanest design context on the internet and still have no idea whether the component that shipped this morning came from your library or got hand-rolled as a div with a hardcoded blue. Murphy Trueman makes the point plainly: the more rules you give an agent, the harder they get to follow. Researchers found the best model followed fewer than a third of instructions in full, and adherence dropped as the instructions grew.

The answer borrowed from teams shipping AI features is evals. An eval is a test for output you can't predict. You write prompts your team would actually type, run them through the agent, and check what comes back. Mechanical checks catch invented props, raw hex values, and missing labels in milliseconds. Model-as-judge rubrics handle the softer calls, like whether the agent reached for a confirmation dialog over a generic modal.

And you run the suite, not a single check, because the ground moves. The model behind the agent gets updated and starts treating your tokens differently. Someone trims three lines from your context file and breaks a behavior nobody re-tests. Meta's Astryx, open-sourced in June, runs the same prompt across different configs and measures how faithfully each rebuilds. Kaelig Deloumeau-Prigent wired an evaluate stage into an eight-agent pipeline for the Intuit Design System, checking tokens and accessibility on every generated component. Trueman's read: most teams have neither grounding nor evals.

A system is a claim someone has to keep

All of this leans on a design system, and the way teams treat that system is shifting. Mandy Michael frames it well: a design system is not a static artifact, it is a continuously maintained claim about what a product should look and behave like. Agents make that claim get exercised a thousand times a day.

Which raises a staffing question. The design system career ladder that Mostafa Esmaeili lays out is blunt about where the senior work actually is. It is not building better components. It is governance: deciding what enters the system, what gets deprecated, and how contribution works. A Design System Manager splits time roughly 30% team, 40% strategy and governance, 30% stakeholder wrangling, and their hardest job is the three-sided fight between design wanting flexibility, engineering wanting stability, and product wanting speed.

That is the role that owns whether your design.md is current and whether your evals still pass. A system nobody maintains is worse than no system at all when an agent is generating against it.

The deep cut

Grounding and evals solve different problems, and you need both. Grounding stops bad output at generation, the way v0 refuses to emit a token it can't verify. Evals catch what slips through and keep catching it as the models drift underneath you. Neither one closes the taste gap. A token-perfect button can sit inside a broken flow, and no suite will flag that.

So the eval suite is not there to remove the reviewer. It clears the mechanical failures off the table so your reviewer spends their attention on judgment instead of hunting for hardcoded colors. On Monday, that means one concrete move: add a regression suite to your design system the same way you ship a changelog, wired into pull requests, and name a person who owns keeping it and the design.md current. Right now that job is nobody's in particular.

Three questions for your team

1. If an agent shipped a component this morning, could we prove it came from our library and used real tokens, or are we guessing?
2. Do we have a design.md that captures our spacing, color logic, deprecated patterns, and accessibility rules, and is it current enough to hand an agent today?
3. Who owns the governance now that agents generate against our system, and does that person have the time split to actually keep the rules and the evals alive?