20VC Annie Duke on Reversible vs Irreversible Decisions, How To Evaluate Risk, The Theory of Sunk Cost in Venture & How to Optimise Both the Discussion and Quality of Investment Decisions

In this episode of 20 VC, host Harry Stebings interviews Annie Duke, a former professional poker player, World Series of Poker bracelet winner, and author of "Thinking in Bets." Duke discusses her journey from cognitive science Ph.D. work to poker and how her experiences have shaped her understanding of decision-making, risk, and critical thinking. She emphasizes the importance of forecasting, pre-mortems, and learning from the dispersion of opinions to avoid biases and sunk cost fallacies in both poker and venture investing. Additionally, the episode touches on the value of collective decision-making in partnerships, the necessity of exploring outside one's comfort zone, and the potential pitfalls of not adequately assessing one's expected value in risk evaluation. Harry also endorses various products and platforms, including AngelList's fund admin platform, Remote's global employment solutions, and Ramp's spend management platform.

• Harry Stebbings expresses excitement for the interview with Annie Duke, a favorite author of his.
• Annie Duke's book "Thinking in Bets" has significantly influenced Harry's views on decision-making.
• Annie is introduced as a former professional poker player with notable achievements and the author of the national bestseller "Thinking in Bets" and "How to Decide".
• Annie founded "How I Decide", a nonprofit focusing on improving decision making and critical thinking for underserved middle schoolers.

"Her book thinking in bets really impacted so much about how I think about decision making, and so I'm thrilled to welcome Annie Duke to the show today."

The quote highlights the profound impact Annie Duke's book had on Harry Stebbings' perspective on decision-making and sets the stage for the interview's focus on decision-making strategies and thinking processes.

• Annie Duke shares her journey from cognitive science to poker, attributing much of her path to luck.
• Initially set on an academic career, a health issue led her to take a break from her Ph.D. work.
• Poker wasn't widely recognized as a lucrative profession at the time, and Annie had a tangential knowledge of the game through her brother.
• Her brother, a chess enthusiast and poker player, introduced her to poker as a means to support herself during her hiatus from graduate school.
• The temporary engagement with poker turned into an 18-year career.

"I was actually doing phd work at the University of Pennsylvania. I was going to become a professor and really had no intention of doing anything else."

This quote explains Annie Duke's original career intentions, which were focused on academia before her unexpected shift towards professional poker playing.

• Annie Duke discusses the lack of widely available information on poker strategy during the 1990s, which gave knowledgeable players an advantage.
• She compares the poker scene of the 90s to the early days of options trading, where knowledge of models like Black-Scholes provided a significant edge.
• The informational gap allowed professionals to achieve success without being exceptionally skilled by today's standards.
• Annie emphasizes that the gap between those who understood the game and those who did not was substantial.

"The gap between the people who understood the game and knew the game and everybody else was so wide that as a professional, you didn't need to be nearly as good then as you would need to be now in order to win."

Annie Duke highlights the significant advantage that knowledge and understanding of the game provided to poker professionals in the past, an advantage that has diminished as information has become more accessible.

• Annie Duke reflects on her perception of risk and how she didn't view the move to poker as particularly risky due to her success and the skill gap in the game at the time.
• She believes in the non-permanence of most decisions, which reduces the perceived risk, as one can pivot to other options if needed.
• Annie almost became an options trader but decided to continue with poker when it began to gain television exposure.
• She advises considering the reversibility of decisions and the availability of alternative options, not just the possibility of returning to a previous choice.

"I don't think about most decisions that I make as permanent. So that's a way to derisk."

The quote emphasizes Annie Duke's approach to decision-making, where she views decisions as non-permanent and believes that this mindset can reduce the perceived risk associated with them.

• Annie Duke suggests that many decisions we consider irreversible are not as final as we think.
• She encourages a deep assessment of whether a decision truly prevents returning to previous options or if it's just a feeling of finality.
• Reversibility can also mean the ability to switch to different options, not necessarily returning to the original situation.
• For decisions that are genuinely hard to reverse, like marriage or having children, Annie acknowledges the difficulty but also suggests maintaining flexibility in thinking about options.

"A lot of decisions that we think are irreversible are actually reversible. I think that we feel like we can't go to options, back to options that we've rejected in the past much more than is actually true of the world."

This quote challenges the common belief that certain decisions are final and encourages reevaluating the true reversibility of our choices.

• Decisions are viewed on a continuum based on the cost to reverse them.
• Higher cost to reverse a decision necessitates a more accurate model of the world to make an informed decision.
• When the cost to reverse is low, the need for an accurate model is less critical.
• Building a good model through smaller, low-cost decisions can improve accuracy for high-cost decisions.
• This approach is similar to poker, where small bets gather information to inform larger, more costly decisions.

"So when the cost to reverse is quite high, what I actually think about is what are the things that I can do beforehand that are going to help me to build a really good model of the world that will make my decision more accurate when the cost of reverse is high."

The quote emphasizes the importance of preparing and building a good model to make accurate decisions when stakes are high and reversibility is costly.

• In venture capital, rapid investment decisions can lead to insufficient information.
• Portfolio theory can mitigate the cost of inaccuracy by spreading bets across multiple investments.
• Sorting at the top of the funnel is crucial to identify positive expected value opportunities.
• Time constraints and uncertainty can prevent distinguishing between good investments.
• Investing across a group of positive expectancy opportunities can be a strategy.

"So there's a couple of solutions that you can do for that. One is another way to mitigate the cost of being less accurate is it's essentially just portfolio theory, right?"

This quote suggests using portfolio theory to reduce the impact of inaccuracies by diversifying investments.

• Collective decision-making can be affected by biases and inconsistencies.
• Breaking down broad judgments into component parts can discipline bias and noise.
• Rating the components before assessing the broader category improves judgment accuracy.
• Eliciting individual opinions before group discussions can enhance decision-making quality.

"One is that when we're making really large judgment, we know that a lot of bias can enter into the process and actually a lot of noise as well."

This quote highlights the challenges of bias and inconsistency in collective decision-making and the need for structured approaches to mitigate them.

• Disciplined judgment involves breaking down decisions into smaller, measurable components.
• Group discussions should be preceded by individual opinion gathering to prevent information loss and groupthink.
• Eliciting opinions in advance allows for a clear view of the group's knowledge and agreement areas.

"So one of the things we want to do is break sort of broad judgments down into their component parts."

The quote stresses the importance of deconstructing complex decisions into smaller parts to reduce bias and improve decision-making accuracy.

"So there's a whole bunch of work by a bunch of different people. There's a study by Thasser, Richard Zeckhauser and Dan Levy at Harvard have done work on this."

This quote points to research supporting the practice of obtaining individual opinions before group discussions to improve the quality of collective decision-making.

• Discussing dispersion of opinion is crucial for effective group decision-making.
• Focus on areas where opinions differ, rather than where they align, to fully utilize the group's potential.
• Encouraging members to articulate their differing views on specific aspects provides a comprehensive understanding of the issue at hand.

Everybody can see the agreement, so you don't need to go and double click on that over and over again. You can say, okay, we generally agree about the market opportunity, but there's a lot of dispersion in the group about the quality of the team.

The quote emphasizes the importance of identifying and exploring areas where group opinions diverge to enhance the decision-making process.

If you only talk about the agreement between the group, why do you have more than one person having the discussion?

This quote highlights the redundancy of group discussions if they only focus on points of agreement, rather than exploring diverse perspectives.

• Premature consensus can shut down information discovery, which is critical for decision-making.
• The goal of group discussions should be to explore different viewpoints and information, not to quickly reach agreement.
• Facilitating a thorough exploration of opinions can prevent the risk of decisions being based on incomplete information.

This consensus, this feeling of consensus that groups are trying to get to, actually builds velocity very quickly. And once people start to sense consensus, it just speeds up.

The quote describes how a group's rush to consensus can lead to hasty decisions without adequate exploration of information.

• The sunk cost fallacy can lead to an escalation of commitment to a failing course of action.
• Recognizing the "dark side" of grit is essential to avoid sticking with decisions or investments beyond the point of rationality.
• Understanding human tendencies and biases can help in creating strategies to counteract them.

Sunk Cost is really kind of the dark side to grit, right. We don't want to stick to things long after we're supposed to.

This quote explains how the sunk cost fallacy can mislead individuals into persisting with unprofitable endeavors due to past investments.

• Conducting a pre-mortem analysis helps anticipate potential problems and prepare for how to react.
• Pre-mortem should consider both decision-making flaws and external factors such as luck.
• Identifying early warning signs and establishing a plan allows for more informed and flexible decision-making.

As soon as you make an investment, you need to do pre mortem and you need to say, what are the ways that this investment can go wrong that both have to do with my own decision making and luck, figure out what those are, what the probabilities of those occurring are, and then you need to do a bunch of different things off of that.

This quote underlines the importance of conducting a thorough analysis of potential failure modes for an investment before problems arise.

• Continuous monitoring allows for the adjustment of strategies based on new information and changing circumstances.
• The pre-mortem document should be revisited and updated regularly to reflect new insights and developments.

The pre mortem doesn't live in stone. You're going to learn new information as you go along.

This quote stresses the need for adaptability in investment strategies, acknowledging that circumstances change and new information must be incorporated.

• Past outcomes can unduly influence future decisions, creating biases towards certain types of investments.
• Partnerships and diverse teams can help balance individual biases and maintain a varied investment portfolio.
• Actively exploring and understanding market opinions on rejected opportunities can provide valuable insights and counteract personal biases.

What I would say is kind of number one, that's a little bit why you have a partnership, because I think there's no doubt that individuals are going to get affected by the things that have worked for them in the past.

This quote highlights the benefit of partnerships in venture capital, where the diverse experiences of partners can mitigate individual biases.

You always have to have some exploratory lines open. Right. You have to say, I'm specifically going to take some meetings... with people that I think are going to be really bad because that's going to actually, is what's going to give you the information.

The speaker advocates for deliberately exploring seemingly unattractive opportunities to gain broader market understanding and to challenge personal preconceptions.

• Emphasize exploration outside of one's comfort zone to avoid stagnation.
• Allocate specific time for exploratory activities to learn about new markets.
• Accept a high failure rate in initial meetings as part of the learning process.
• Continuously evaluate the reasons behind rejecting certain opportunities.

"You need to spend some time specifically and set it aside and say, I'm going to take first meetings. A certain percentage of my first meetings I think are going to have a high failure rate, but it's so that I can learn those markets and understand them."

This quote highlights the importance of dedicating time to explore unfamiliar markets despite expecting a high rate of initial failures, as it can lead to discovering valuable opportunities.

• Elicit opinions before group discussions to broaden perspectives.
• Recognize that others may have vastly different views and realities.
• Adopt a forecasting mindset to focus on accurate models of the future.
• Be information hungry and open to the possibility of being wrong.

"So just that little trick is really huge, right? Because now I'm going to get more of that dispersion out of you."

This quote suggests that by asking for opinions before revealing one's own, you can gather a wider range of perspectives, which aids in expanding your mindset.

• Forecasting prioritizes an accurate understanding of the future over personal identity tied to beliefs.
• Good forecasters seek out information and challenge their own judgments.
• Engaging in forecasting can lead to a more polymathic approach to learning and decision-making.

"But what I care about is that I have some accurate model of how the world is going to unfold."

Annie Duke emphasizes the value of striving for an accurate prediction of future events rather than clinging to one's current beliefs, which is crucial for effective forecasting.

• "Superforecasting" by Phil Tetlock is recommended for understanding forecasting.
• "The Success Equation" by Michael Mobison is suggested for exploring luck and skill.

"The book that I always try to get people to read is the success equation from Michael Mobison."

Annie Duke recommends "The Success Equation" for its in-depth analysis of luck and skill, which she considers vital for understanding success.

• People focus too much on calculating risk without assessing if they have a positive expected value.
• It's crucial to determine if one is winning before considering bet sizing and portfolio breadth.

"I think that we can get caught up in discussions about what should our bet sizing be and how broad should our portfolio be and whatnot. And we don't circle back to say, are we actually winning in the first place enough."

Annie Duke points out that the preoccupation with risk management often overshadows the fundamental assessment of whether the overall strategy is successful.

• Acknowledges difficulty in declining requests and talking too fast as weaknesses.
• Identifies curiosity as a significant strength, driving deep exploration of interests.

"I think my biggest strength is that I think I'm curious."

The quote reveals that Annie Duke considers her curiosity to be her greatest strength, leading her to pursue various interests deeply.

• Prefers to invest alongside more knowledgeable individuals in venture and stock markets.
• Uses indexing for her stock portfolio due to the expertise of others in the market.
• Values the opinions of top-tier investors and emerging funds like Renegade Partners.

"I have done angel investing, but it's always alongside. So I'm really betting somebody else's opinion who I think is much smarter than I am."

Annie Duke shares her strategy for angel investing, which involves collaborating with investors she considers more knowledgeable in the field.

• Working on two books and consulting in decision strategy.
• Participating in research with Phil Tetlock's lab, potentially completing her Ph.D.
• Enjoys playing tennis and staying informed through reading and watching series like "Queen's Gambit."
• Remains open to pivoting to areas that allow for deep exploration of uncertainty.

"I've got a pretty good idea of what the next two years is going to look like."

Annie Duke outlines her plans for the near future, which include a mix of professional and personal activities centered around her interest in decision-making under uncertainty.