Vercel: OpenAI's crawler ran zero JavaScript across 500 million requests

There's a new job forming on your team, and nobody has the title yet. It's about whether ChatGPT, Claude, and Google's AI Overviews can find your product, describe it right, and cite you when a buyer asks. The money is already moving. The tactics are already splitting into winners and losers. Let me catch you up.

The deep cut

• AI search reads your server, not your browser. OpenAI's crawler ran zero JavaScript across 500 million requests Vercel measured.
• Give the model something it can only get from you. Karo's Pangram experiment got cited because no other page had that result.
• A wrong answer is your first impression. Claude called DraftKit a scheduling tool it never had.

The market voted before you did

Profound just raised a $180 million Series D at a $1.8 billion valuation, seven months after its last round. Revenue tripled in six months. Its customer list includes Comcast, Estée Lauder, and Walmart. That is not a science project. That is enterprise budget flowing into one question: are we showing up in AI answers?

The reason the budget exists is a shift in where buyers start. Gartner projected traditional search volume dropping 25% by 2026 and 50% or more by 2028, per the endpoint teardown from Elena. Same piece cites Mersel AI: the overlap between top-10 Google rankings and AI Overview citations fell from 75% in mid-2025 to between 17% and 38% by early 2026. A great Google rank no longer means the AI knows you exist.

Your fast site might be an empty page to a bot

Here is the part that stings. Modern sites build the page in the visitor's browser using JavaScript. Humans see it fine. AI crawlers do not run that code. They grab the first response, take whatever came back, and leave. Vercel and MERJ went through 500 million requests from OpenAI's crawler and found no sign it ever ran the JavaScript. Anthropic, Perplexity, and Meta behave the same way.

The scale is real. About 32% of content pages need a look. Around 9% serve under half their content to bots. If your prices and product copy only appear after the browser does its work, the AI is reading a blank frame. The fix is server-side rendering: send a finished page. It may feel slower to load, but the machine gets the whole thing.

There's a second leak. If your links get built in the browser too, the crawler hits a dead end and stops. One in fifteen pages is a dead end, and 169,377 pages serve their text perfectly while offering no crawlable links. The bot can't tell a real dead end from missing navigation, so it just leaves.

The file everyone recommends barely moves the needle

You have heard the advice: add an llms.txt file. Karo believed it too, and put real work into hers. Then the data came in. Ahrefs checked 137,210 domains and found 97% of valid llms.txt files got zero requests in May 2026. SE Ranking ran 300,000 domains and found that removing llms.txt made its model more accurate. Google's own guide says you don't need machine-readable files because Search doesn't use them.

What actually got Karo cited was information gain. Her Pangram test gave the model something no other page had. Her rule is plain: before publishing, ask whether the page holds a fact, result, or opinion an answer engine could only get from you. If AI can make the same point without you, it will. Over 60 days, Perplexity and Claude drove 51 subscribers and $953.40 in tracked revenue. Not a media empire. A checklist.

Feed the model, don't hope it guesses

Elena ran the test most builders skip. She asked three AI tools about her category, then named DraftKit directly. Claude called it "a newsletter scheduling tool with collaboration features." DraftKit has no scheduling feature. The model had locked onto an old blog post and would not let go. A 4,200-word landing page written for humans and Google was not reaching the models at all.

Her fix was a machine-readable layer: a JSON-LD endpoint at /api/v1/agent-info using Schema.org, a hidden plain-English brief on the page, an llms.txt file, a robots.txt update to allow GPTBot and ClaudeBot, and the same product vocabulary across all four. Consistency turns scattered signals into a confident description. She points to Eric Linssen, whose SaaS product went from 575 to 3,700 monthly trials in seven weeks after rebuilding its AI search presence. The work sat under the product page, not on it.

Serving two versions opens an old trap

Cloudflare turned on plain-text and markdown delivery in February 2026, so a crawler that asks for markdown gets a stripped-down page. Cloudflare's numbers put a blog post at about 16,180 tokens as a web page and 3,150 as plain text, roughly an 80% saving. Fastly and Vercel shipped their own versions. Hardly anyone uses it yet.

The moment you serve one thing to people and another to machines, you've recreated the conditions for cloaking. Google has spent years policing exactly that. And if your site is client-rendered, the CDN faithfully converts your empty frame into empty markdown. A clean rendering of nothing. Fix the server-side rendering first, or the fancy new pipeline just hands the bot a better-formatted blank.

Three questions for your team

• If we ask ChatGPT and Claude to describe our product today, what do they say, and where did they pull it from? Run Elena's test this week.
• Does our site serve the full page and its links in the first server response, or are we shipping blanks to crawlers that never run JavaScript?
• Before we ship our next page, can we name one fact, result, or opinion on it that an answer engine could only get from us?