Microsoft's Ryan Roslansky: summarizing an AI doc with AI is a "doom loop

The money got spent. The output came back thin. Now the budgets are getting cut, and the leaders who swore AI would replace half the org are walking it back in public. If you run a design or product team, this is the moment where the air comes out of the balloon and you get to set real expectations. Let me catch you up.

The deep cut

• Measure shipped outcomes, not activity. PostHog's chat tool looked busy while its MCP server pulled 5x more daily users.
• Slop costs more than nothing. Microsoft's Ryan Roslansky called summarizing an AI doc with AI a "doom loop."
• Shadow AI shows where your tools fail. Locked-down Copilot pushed Paul Boag's UX team to personal ChatGPT accounts.

The slop tax nobody budgeted for

The pullback is real and it is coming from the companies selling the dream. Reuters reported that Meta is scaling back plans to replace staff with AI agents, and Microsoft is doing the same. Microsoft's own EVP Ryan Roslansky posted on LinkedIn that someone sent him a clearly AI-generated doc, he had his AI summarize it, and "ended up with nothing that brought new thinking." He called it a doom loop.

The damage is not just wasted tokens. An Ernst and Young survey found 36 percent of senior AI executives at billion-dollar firms had already hit a "catastrophic AI incident" with real financial or brand cost. The SHRM CEO put it plainly: "we're driving the truck and building it at the same time."

Your takeaway: the slop tax is a line item now. When someone pitches an AI rollout in your next review, ask what it costs when the output is wrong, not just when it works.

Count what ships, not what runs

The teams getting value stopped counting activity. PostHog's 38 lessons from building ten AI products land on one point: a generic chat assistant bolted next to your features is weak. Their MCP server, which lets agents take real actions, pulled five times more daily users than the built-in chat. Their best win was an onboarding wizard that cut setup from two hours to eight minutes and doubled activation.

Claire Vo's ROI framework in the same digest says leaders waste attention on small departmental pilots while engineering eats over 80 percent of token spend. Pull request velocity alone lies. Time saved does not become money unless freed hours go to growth work.

So instrument the outcome. Tie model usage to shipped projects and check the session logs. A dashboard of prompts is not proof of value.

When you lock it down, it goes underground

Paul Boag's piece in Smashing Magazine names the trap most IT teams walk into. They roll out one tool like Copilot company-wide, call it done, then write pages of rules about what staff cannot share. The result is shadow AI: people open personal ChatGPT or Claude accounts on their phones because the official tool cannot do the job. Company data lands in places you cannot see.

Boag's read is that shadow AI is a signal, not a crime. It shows exactly where your tools fall short. He describes a UX team stuck waiting on an AI steering group because Copilot cannot build the research repository they want. Their enthusiasm is draining while they fight the committee.

His fix: a short set of principles that fit on a poster, trust in common sense, and room to fail without a reprimand over token counts.

The real bottleneck moved

Ed Biden, a former CPO now teaching product, argues in The CPO Club that the hard part is no longer building. His small non-technical team killed its agency and does platform migrations in-house with Claude. He cites Johnny Quach at MotorK, a fifteen-year-old SaaS company that killed sprints and Jira, writes tickets straight in Cursor, and shrank teams by 30 to 35 percent. Capacity jumped five to ten times in months.

When capacity moves that fast, sprints and story points become friction, not coordination. Biden's sharpest line: the bottleneck moved from "Can we build it?" to "Do we know what's worth building?"

That is the reframe worth bringing to your team. If building got cheap, your edge is judgment about what deserves to exist. Biden keeps customer calls, editorial voice, and conviction explicitly human. The machine executes; a person owns the call.

Three questions for your team

• What does a wrong AI output cost us, and where would a "catastrophic incident" actually hit? Price the downside before the next budget ask.
• Which shipped projects can we tie to AI usage this quarter, and are the freed hours going to growth or just disappearing? If you cannot name them, you are measuring activity.
• Where is shadow AI already happening on this team, and what does it tell us our official tools cannot do? Treat it as research, not a violation.