The Knowledge Project — selected conversations
Gary Klein: Building expertise and making better decisions
Five selected conversations on product judgment, decision-making, and AI adoption: Geoffrey Cain on Steve Jobs's NeXT years, Benedict Evans on technology shifts, Bret Taylor on AI products, Patrick Collison on building Stripe, and Gary Klein on expertise and pre-mortems. Attributed experiences, proposals, and forecasts—not validated management prescriptions.
In this collection Browse 5 summaries 5 of 5
Cognitive psychologist Gary Klein talks with Shane Parrish about developing expertise rather than simply collecting decision-making rules. His emphasis is on noticing unexpected evidence, understanding how a situation works, and learning what experienced practitioners see that others miss.
Based on English auto-generated captions, without an original-audio check. Research results and workplace examples are Klein's account; this summary does not independently evaluate the studies. The date above is the video's YouTube publication date.
Key ideas
- Reducing errors and discovering better approaches are different goals. Klein argues that organizations often reward predictability because mistakes are visible while missed insights are not. That can discourage investigating anomalies and pursuing new ideas that might improve performance. His suggested response is selective curiosity: briefly investigate something surprising rather than automatically explaining it away, without trying to pursue every anomaly. 01:46–11:31(opens in a new tab)
- Experience is not the same as expertise. Repeating a job for years does not guarantee learning from it. Klein suggests combining several imperfect signals: ask about recent mistakes, examine reasoning in realistic scenarios, consider track records, and ask peers whom they rely on. Confident presentation and successful outcomes alone can mislead; a good result may be luck. 14:27–23:01(opens in a new tab)
- A useful mental model includes more than normal operation. Klein's model covers how a system works, where it breaks down, possible workarounds, and where other people may become confused. Asking an expert what they noticed, expected, and found surprising in a specific incident reveals more of that knowledge than a polished general rule. 23:01–23:53(opens in a new tab), 29:51–35:48(opens in a new tab)
- Preserve what was known before the outcome. Parrish proposes a decision journal; Klein suggests recording the decision, goals, important information, and affected people or teams. Later, examine both reasoning and outcome: a sensible decision can fail, and a poor one can succeed. Klein also proposes tracking evidence inconsistent with the initial interpretation so the review can ask whether it should have been revised sooner. 23:53–27:44(opens in a new tab), 40:25–41:21(opens in a new tab)
- Practice interpreting difficult situations, not just naming the method. Klein describes recognition-primed decisions as a blend of patterns learned through experience and imagining how an option would play out. His ShadowBox training uses scenarios where learners rank actions, goals, or information and explain why, then compare their reasoning with recorded expert explanations. The learning target is what experts noticed and inferred, not merely matching their selected answer. 1:24:50–1:33:48(opens in a new tab)
Running a pre-mortem
Klein's pre-mortem makes it easier to raise concerns before a project begins. Instead of asking vaguely what might go wrong, the team assumes a future failure and explains it. In his account, surfacing a credible problem becomes a contribution rather than an objection to a plan everyone has just endorsed. 50:44–1:00:56(opens in a new tab)
- After explaining the plan, choose a future point and ask the team to imagine that the project has already failed badly.
- Give everyone two minutes to privately write reasons why the imagined failure happened.
- Have the leader offer a genuine concern first. Go around the group, recording one new reason per person per round; avoid letting one speaker deliver their whole list.
- Give everyone another two minutes to write what they personally can do to reduce the listed risks.
Applying the discussion carefully
Editorial implications: use incident scenarios to expose gaps in understanding without deliberately creating real failures. Separate learning exercises from high-stakes personnel evaluation: Klein recounts a confident petrochemical worker failing to understand the plant's dynamics; after his panel assignment was deferred, others avoided the training. 17:27–20:07(opens in a new tab), 44:12–48:55(opens in a new tab)
Klein questions the effectiveness of debiasing—efforts to reduce cognitive biases in judgment—and explicitly identifies his view as an outlier position. 1:00:56–1:05:26(opens in a new tab)
Editorial caution: his defense of experience-based rules of thumb is not evidence that intuition is always reliable. Elsewhere he emphasizes noticing contradictions and revising an initial impression rather than protecting it. Likewise, matching expert rankings in training is not the same as demonstrating better outcomes under real-world pressure. 38:31–41:21(opens in a new tab), 1:30:17–1:34:38(opens in a new tab)