Confirmation bias in decision-making is the habit of trusting a number because it feels complete, not because anyone tested it. Awareness doesn't stop it — a clean calculation still feels like proof. The fix is structural: before you commit, name the one fact that would break the number, then go check it.
Thirty thousand email addresses felt like proof.
Confirmation bias in decision-making rarely feels like bias. It feels like certainty — and the numbers backed me completely. The law of large numbers said we couldn't lose: a low conversion rate would still hand us 300 customers, an exceptional one, 3,000.
I put both numbers on the whiteboard in front of the team. Even the conservative outcome looked like a win, and nobody in the room found a reason to disagree with me — including me. It felt less like a decision than a formality.
So I told Mike we couldn't fail, and we rebuilt the business model on that number.
The maths was way off.
Not the arithmetic. The assumption underneath it — I'd never asked whether those 30,000 addresses were people who wanted to hear from us, only whether there were enough of them.
Mike had bought the list through a Facebook ad. Nobody had ever engaged with it. The quality thinned, the conversion rate died, and our first campaign closed with single-digit subscribers.
How Does Confirmation Bias Affect Decision-Making Before You've Even Made One?
We're taught to question our assumptions. Almost nobody teaches us to question the number that agrees with us.
This bias doesn't usually look like ignoring evidence. It looks like accepting the first calculation that supports the decision you already wanted to make, then treating that calculation as due diligence. The email list didn't fail because I skipped the analysis — it failed because the analysis I did was the only one I needed to hear.
Confirmation bias rarely works alone. It sits inside a wider set of cognitive biases that shape decisions long before anyone notices they're deciding at all, and it's usually the first one to show up once a number already looks like the answer you wanted.
That's worth naming plainly — leaders rarely think of themselves as biased, they think of themselves as thorough. The maths on the whiteboard felt like thoroughness. It was actually the fastest route to a conclusion I'd already reached.
The real problem isn't a bad calculation. It's a good one, arriving too early.
Don't Let a Good Number Make the Call for You.
One Good Decision gives you the one-fact test to run before you commit — applied to a real decision you're facing — for free.
What Does the Research Say About Mistaking Volume for Evidence?
Daniel Kahneman has a name for this: What You See Is All There Is.
In Thinking, Fast and Slow, Kahneman describes a mind that runs on two systems — one fast and intuitive, one slow and deliberate. The fast system doesn't fail by ignoring evidence. It fails by building a complete-feeling story from whatever evidence shows up first, then handing that story to the slow system as fact.
The slow system is supposed to check it. What it actually does is confirmation bias: not testing whether the story is true, but hunting for reasons to believe it already is. That's a different failure from survivorship bias, which doesn't hunt for confirming reasons at all — it just never notices the missing cases in the first place.
Thirty thousand addresses were a coherent story. The pattern rarely announces itself as a gap. It arrives as a number that already adds up.
The dangerous part isn't the missing information. It's that missing information doesn't feel missing. Nothing in the calculation flagged that engagement mattered, so nothing prompted the question.
This is why "look harder" is bad advice for this specific trap. I wasn't failing to look. I was looking at genuinely real numbers, correctly calculated, and mistaking their coherence for completeness.
The fix isn't more scrutiny of the number in front of you. It's a deliberate habit of asking what number isn't in front of you yet — before the story built from the first one gets to make the decision for you.
How Do You Tell the Difference Between Evidence and Proof?
Evidence and proof are not the same thing.
Evidence supports a conclusion. Proof survives an attempt to break it. A number that predicts 300 to 3,000 new customers is evidence, not proof, until someone has tried to break it.
Most business decisions never make it to proof. They stop at evidence, because evidence already feels sufficient. That's how confirmation bias in decision-making actually operates — the moment a number confirms what you wanted to believe, the search for anything that would break it quietly ends.
This is the practical answer to WYSIATI. If the mind treats a coherent story as complete, the only way to stop it is to build the missing piece into the story before it's told, not after.
Name one fact, not every fact. The test is the fact whose absence would mean the whole number falls apart, checked before the decision, not after.
A sales forecast built on lead volume alone is evidence. The breaking fact is lead quality — how many of those leads have ever opened an email, clicked a link, or shown any real sign of intent. Test that fact before the model, not after.
The same test runs on a hiring decision. A strong interview and a glowing reference feel like proof you've found the right person, but they're evidence you liked them, nothing more. The breaking fact is narrower — has this person actually done this specific job, under these specific constraints, before.
A great interview with no track record on the actual task is a story, not a guarantee.
This test is uncomfortable to run out loud. Naming the one fact that could break a plan can sound like negativity, or like overthinking a decision everyone else in the room is ready to make.
Ask it as a question instead of an objection: what would have to be true for this number to be wrong? That framing invites the check without forcing anyone to defend against being the pessimist.
Most forecasts, and most hires, fail this test before they're ever made. Not because the information wasn't available, but because nobody asked what fact would need to be true for the conclusion to hold, and nobody went looking for it.
What Would Have Caught This Before It Cost the Campaign?
The breaking fact was sitting in the list the whole time.
Thirty thousand addresses, bought through a Facebook ad, with no history of opening anything we'd sent. Not one had clicked a link, replied to a message, or shown any sign a real person was reading what we sent. That fact existed before Mike and I ever ran the numbers.
Run the test from the framework against that decision, and it takes one sentence: what fact would have to be true for 30,000 addresses to become 300 customers? The answer is engagement, not volume — and none of that had been checked.
Checking it would have taken an afternoon. Pull a sample of a few hundred addresses, send one real email, and look at the open rate before rebuilding a company's strategy around the response.
If the open rate had come back near zero, the whole calculation would have collapsed before a dollar was spent on the pivot. If it had held up, the confidence would have been earned instead of assumed.
That afternoon test is the first thing I run now, before any number gets to shape a decision. Not a full audit, not a committee — just the one fact that could break the story, checked before the story gets told to anyone else.
I checked the maths. I never checked the fact the maths depended on.
That's the entire gap between evidence and proof, played out with a real business and a real cost. It's the same gap sitting inside whatever number is currently making a decision feel safe for you.
Certainty is not a signal that you're right. It's a signal that you've stopped checking.
Most advice on confirmation bias tells you to go looking for disconfirming evidence. That's true, and almost useless — nobody has the time to disprove every number they're handed on a Tuesday.
The realistic skill isn't an open-ended search. It's naming the one fact your decision depends on, and checking that fact before you commit to the number.
What number are you trusting right now that you haven't actually tested?
FAQs
Does more data reduce confirmation bias in decision-making?
No — more data collected the same way doesn't fix confirmation bias in decision-making. The fast-thinking part of the mind still builds one coherent story from whatever it's given, and a bigger data set just makes that story feel more convincing. The fix is testing the story, not expanding it.
How do I know if confirmation bias is affecting a decision I'm making right now?
If you can't name one fact that would prove your own plan wrong, that's the signal. Confidence isn't evidence you've checked enough — it's often evidence you've stopped looking. Ask what fact would break the number before you commit to it.
What's the difference between confirmation bias and WYSIATI?
WYSIATI is the mechanism — the mind treating whatever evidence is visible as the whole picture. Confirmation bias is what happens next: instead of testing that picture, the mind searches for reasons to believe it's already true. One builds the story, the other defends it.
What's a simple way to start testing decisions instead of just researching them?
Before committing to any number, name the one fact that would collapse it, then go check whether that fact is true — the same discipline behind worst case scenario planning, just applied earlier, before the decision is made rather than while it's already in motion.
Is confirmation bias the only bias that distorts a leader's decisions?
No — it's one of several. Recency bias, for example, works differently: it's not about favouring confirming evidence but about over-weighting whatever happened most recently, even when it isn't the strongest evidence in the room. The mechanism differs, but the fix is the same instinct — test the input before it decides for you.
Don't Let a Good Number Make the Call for You.
One Good Decision gives you the one-fact test to run before you commit — applied to a real decision you're facing — for free.
Written by
