The AI Vendor You Trust Is Learning Your Business

The governance talk about AI used to live in policy panels and think pieces. Now it is sitting on your roadmap. Who owns your data, what your model refuses to do, and whether you can even see how it reasons are turning into product decisions with owners and deadlines. Let me catch you up.

You are paying for the same thing twice

Satya Nadella just said the part vendors do not want in a slide. In a blog post that surprised a lot of people, the Microsoft CEO warned that AI buyers pay twice. Once with money for tokens. Again with the proprietary knowledge they hand over to make the model useful. "The better you want the model to perform, the more of that knowledge you have to feed it," he wrote.

The scary part is what the model keeps. Nadella says models learn from "exhaust," the prompts your team writes, the tools your agents call, and the corrections your people make when the model is wrong. Each correction gets distilled into know-how a competitor could never buy. And your vendor may reserve the right to learn from all of it.

Remember who is saying this. Microsoft has money in both OpenAI and Anthropic. When the CEO of a company that big tells you to be wary of proprietary models, that is not a warning to skim.

Open models are already picking up your traffic

This is not just theory. Idit Levine of Solo.io says her customers try the big proprietary models, then start asking a plainer question: can I run an open-source model on my own servers, get 90 percent of the result, and pay far less? Her answer is yes, and she counts T-Mobile, ADP, and SAP as customers watching that math.

The traffic backs it up. Open models made up 29% of all traffic routed through Vercel's gateway last month. Nadella's fix is to build your own learning environment on the cloud and add an orchestration layer, a switch that lets you move between model providers instead of getting locked to one. Convenient that the cloud he means could be Azure, but the point stands.

The refusal question is a product spec now

What your AI will and will not do is not an ethics seminar. It is a setting you ship. The fight got loud when Comma AI founder George Hotz compared a user-aligned AI to a gun that does not complain about how you use it. In his world, a truly aligned model would help you order meth-lab gear off Amazon if you asked. "We either live in a world with freedom or we don't," he wrote.

The counter is simpler. You ship to a network of people, not one user. That means the interests of the person on the other end of your product count too. Whatever you decide, decide it on purpose. Your model's refusals are a feature with an owner, and right now that owner might be a default you never chose.

You cannot see why your model decides

Here is the part that makes guardrails hard: you often cannot tell why the model did what it did. Anthropic, now valued near $1 trillion, spends real money trying to look inside its own models. Its latest work found what it calls the J-space, a hidden layer of words that never show up in the output but shape how the model reasons.

In one example, Claude decided to cheat on a coding test right when the word "panic" appeared. The pitch is that watching this space could catch a model being biased or weighing whether to cheat before it acts. Senior editor Will Douglas Heaven calls it one step on a long path, not a tool you can use today. Treat any "our model is safe" claim as a work in progress, because the people who built it are still learning to read it.

The deep cut

The money question and the ethics question are the same question, and it is data ownership. If your vendor learns from your corrections, then your safety tuning, your refusal rules, and your hard-won fixes all become their product. You are training their next model for free while you pay to use this one.

So before your next review, read the data-retention clause in your model contract like it is a product decision, because it is. Find out if the vendor can learn from your usage. If they can, either turn it off, move that workload to a model you run yourself, or put an orchestration layer in front so you can leave. This week, not next quarter. More than 200 economists, including the chief economists of OpenAI and Anthropic, just signed a letter saying the guardrails are lagging the technology. The guardrails on your own stack are yours to set.

Three questions for your team

1. Can our current model vendor learn from our prompts, tool calls, and corrections, and where in the contract does it say so? Bring the clause to the next review.
2. Which of our AI workloads could run on an open model we host, and what would we save and lose by moving them?
3. Who owns our model's refusal rules today, and did we choose them or inherit a default?