Amplitude Just Handed the Analytics Loop to Agents. Where Do You Still Hold the Pen?

Amplitude has been shipping AI agents at a steady clip. Agents that read your code and write tracking calls. Agents that run a feature launch from a Slack channel. Agents that dig through session replays and file the ticket before you knew there was a problem. The pitch is the same across all of it: the agent does the moving, you do the judging.

That sounds nice. It also moves real decisions off your plate, and some of those decisions you may not want to give up. Let me catch you up on what shipped, what it actually buys you, and where you still need a human hand on the wheel.

The grunt work is gone, and that part is real

Start with the boring wins, because they are the most solid. The Wizard CLI reads your repo, proposes events named in your product's own vocabulary, and writes the calls inline once you approve. It even hits the ingestion API and waits until the first event lands before it says done. Instrumentation has been the place teams get stuck for years. An agent that does it end to end is worth the install.

The same logic runs through Agent Connectors, where one agent reads session replay, cross-references the GitHub error, and posts the write-up to Slack from a single prompt. AI Engineer Amogh Dikshit runs bug investigations this way and never opens any of the three tools himself.

The honest takeaway from the Wizard team: good agents are mostly about what you don't give them. They narrowed each subagent's tool set, and hallucinated git calls dropped to zero. That is a build lesson, but it is also a buying lesson. Narrow agents work. Open-ended ones drift.

What changes is which bugs get fixed at all

Here is the part that matters more than speed. Amplitude MCP pulls session replay into Claude and Cursor, so the investigation happens in the same place the agent writes the fix. The team is blunt about why that counts: "If investigation takes an hour, you're only ever prioritizing the highest tickets in your queue. If it takes ten minutes, you can work through a class of bugs you used to always defer."

That is a roadmap shift, not a tooling shift. The backlog you've been ignoring because each fix cost half a day of context switching just got cheaper to clear. A replay agent also runs in the background, flags friction patterns, and posts a weekly summary before anything becomes a ticket.

So the question for your next review changes. It is not "are we faster." It is "what work was previously not worth doing that now is, and who decides we should do it?"

When the agent runs the launch

This is where I'd slow down. Amplitude's feature launch setup puts the whole rollout loop in a Slack channel. Agents answer data questions in threads, surface replays when numbers look off, and ramp feature flags without pulling in engineering. You can tell it "ramp the checkout-v2 flag to 25%" right in the thread.

That is convenient and a little scary. Ramping exposure to more users is a judgment call about risk, and now it's one typed command away. The setup does build in checkpoints, where the data at 10% tells you whether 50% is safe. Good. But the agent makes acting feel as light as asking, and those are not the same weight.

Draw the line before launch day. Decide which ramps a human approves and which an agent can execute on a clean metric. Write it down. Do not let the ease of the prompt set your risk policy by accident.

What a forever-loop actually teaches you

The sharpest signal comes from a one-week experiment. Data scientist Eric Carlson pointed a Ralph loop at his own app, refused to intervene, and came back to 102 shipped features, including an avalanche runout simulator and a mushroom foraging model he never asked for. The agent even instrumented itself, so each cycle had behavioral evidence to learn from.

His three lessons are the ones to steal. The loop is not the interesting part; the verification gate and the feedback signal are. Self-instrumentation is what makes it compound. And the bottleneck moves: "When one agent can ship 102 verified features in a week, execution stops being the scarce resource. Taste moves to the front."

Note what he kept his hand on. Merge. Small UI changes could auto-merge; anything touching user data, he reviewed. The safe-to-automate call was per type, not one global switch. That is the model.

The deep cut

The pattern across all of this is that the agent collapses the cost of acting, not just the cost of knowing. Ramping a flag, merging a PR, filing a ticket, these used to carry friction that doubled as a checkpoint. The friction is gone now. So you have to put the checkpoint back on purpose.

Mercado Libre is the proof that this scales. They put Amplitude in front of 10,000 employees, and 70 users built 126 agents, with 71% completing real value actions. But notice the line they hold: "The dashboard remains the canonical source of truth." Agents distribute and act on that truth. They don't get to redefine it.

Do the same. Pick your one or two governed sources of truth, let agents move freely on top of them, and gate the actions that touch users or money by type. The win is not that agents decide more. It is that you spend your attention on which opportunities are worth it, not on copy-paste between tabs.

Three questions for your team

1. Which agent actions touch users or revenue, and which of those still need a human approval before they execute? Write the list before you turn anything on.

2. If investigation just got ten times cheaper, what class of deferred bugs are we now going to clear, and who owns that call instead of letting the backlog reset itself?

3. What is our one governed source of truth that agents read from but cannot redefine, and is every team actually pointed at it?