Validation Is a Spectrum, Not a Checkbox

Someone on your team says the idea is "validated." You nod. You fund it. Six months later it flops, and you find out "validated" meant five customer interviews where people said they liked it. The word did a lot of work it was never meant to do.

The fix is to stop treating validation as a yes or no stamp. It is a spectrum. Your job as the leader is to ask what proof the team actually has, how strong it is, and how much of it. That one shift changes how your team decides what to build.

The deep cut

• Evidence sits on a spectrum, not a switch. Strategyzer scores risk from 0 to 5 because "validated" is a judgment call, not a checkbox.
• What people say is the weakest proof there is. Logitech ran lab tests, shipped Google TV, and lost $100 million to real-world failure.
• Write a fail condition before you run the test. Kromatic adds it to force a decision when the data looks less rosy than you hoped.

Why "validated" means nothing until you define it

When a leader asks "is it validated?" the team hears a demand for a green light, so they find a way to say yes. Lucy Luo at Strategyzer calls this a point of contention, because the insights sit on a spectrum and there is no clean "validated" signal to point to.

That spectrum runs from discovery to validation. Discovery is open-ended and cheap: you go from no evidence to light evidence, usually what people say in a lab setting where they know it is a test. Validation comes later: you go from some evidence to strong evidence, based on what people actually do in the real market.

So change the question. Instead of "is it validated," ask "what evidence did you collect, and how confident are you in it?" Same goal, better conversation.

Separate what people say from what people do

The single most useful cut in all of this: what people say versus what people do. Interviews are fast and cheap, but they only tell you opinions. Watching someone actually behave is always stronger. Luo even flags the question wording. "Would you buy this?" gets you an opinion. "When did you last buy something like this?" gets you a fact.

Investment is another tell. A landing page that collects emails is weak proof of interest. Tesla asked for deposits instead and gathered 325,000 reservations for the Model 3 in one week. Money on the table beats a mailing list.

And the test setup matters. Customers who know they are being tested act differently. Logitech ran extensive lab experiments on Google TV, shipped it, and lost $100 million when the real world did not match the lab.

Score the strength, then count the data points

Strength and volume are two different things, and teams mix them up. Luo's Innovation Project Scorecard separates them on purpose. First, the type of evidence sets your ceiling. Interviews cap out low no matter what, because they are still what people say. Second, the number of data points tells you how confident to feel inside that ceiling. Sixty percent of five people is noise. Sixty percent of a hundred people is a signal worth acting on.

Tristan Kromer at Kromatic makes the same point with sharper teeth. Do not get sucked in by an 80 percent conversion rate when five people were tested and four of them live in your house. Keep the data and the method that produced it together, so anyone can inspect how the number was made.

Design the experiment to fail, not to flatter

Most teams design tests that confirm what they already want to believe. Kromer's answer is the fail condition: pick one metric and one line it must clear, written down before you run anything. You cannot prove a hypothesis true, only fail to disprove it, so set the bar that would kill the idea and see if it survives.

Start with the learning goal, not the build. James Birchler argues the loop is really Learn-Build-Measure, because building without a clear hypothesis is guessing dressed up as science. His template is plain: because we believe X, if we do Y, we expect Z. If your team cannot write that sentence, they are not ready to run the test.

One caution from Kromer: do not hand your team a template and call it thinking. Every field exists to beat a specific bias. Keep the ones that fit your context and cut the rest.

Three questions for your team

• What evidence do we actually have for this, and is it what people said or what they did? Use the answer to decide whether this is discovery or real validation.
• What is the fail condition, and did we write it down before we ran the test? If not, you are reading tea leaves, not data.
• Can we write the hypothesis as "because we believe X, if we do Y, we expect Z"? If the sentence does not hold, go learn more before you build.