William Green (00:02:40):
It’s a great pleasure. I wanted to start by asking you about a pioneering psychology professor named Lila Gleitman, who died last year-

Annie Duke (00:02:50):
Oh.

William Green (00:02:50):
… at the age of 92, because at the very end of the acknowledgment section of your book, you write, “One last final thank you to the late Lila Gleitman, my mentor and best friend. Right up until the week she passed, Lila would inquire about this project, excited about the topic and eagre to be my thought partner. A mentor’s work lives on through their students, and I hope she would have been proud of the finished project. I miss her every day.”

William Green (00:03:13):
And so I was wondering if you could just start by telling us who Lila Gleitman was, and how she influenced you so profoundly?

Annie Duke (00:03:20):
You see, now I’m tearing up. Yeah, so, Lila was just an intellectual giant in cognitive psychology. Her specialty was first language acquisition, she did psycho-linguistics, and she started way back when it was a very interesting time to be a woman in academics. She studied at Penn, she was very good friends with Noam Chomsky, very much under the influence of him, Zellig Harris was her advisor, who these are all obviously giants in psycho-linguistics. And she originally couldn’t get a job as a professor because she ended up marrying Henry Gleitman, who also was just larger than life. He died about 10 years ago.

Annie Duke (00:04:00):
And Henry had a professorship at Swarthmore and so she could not get one, because you weren’t… spouses weren’t allowed to be at the same place. She ended up being a research assistant for a while, before she actually got a tenure track position. But she was a real pioneer in thinking about something really important about… I mean, really think about this as informing, the way that I thought about poker, which is, how does a child learn their first language? Which is not trivial, right?

Annie Duke (00:04:25):
I think that we think that well, people say words and then they point to things, and then obviously if I say book and I point to a book, as the child you go, “Oh, that’s a book,” and then that’s sort of how it proceeds. But it’s actually, if you start thinking about it, it’s like that seems incredibly hard to do because there are all sorts of things which first of all, are non-obvious about pointing to it. Like, how would a child ever learn a word like think?

Annie Duke (00:04:50):
So you point to someone who’s thinking, what are you pointing at, right? Or believe, that doesn’t exist in the world, it’s conceptual, right? And then you can also think about other types of even action verbs, like walk. Which is really hard, because when you point to the action of walking, you’re also pointing to lots of things. And that’s separate and apart from the structure of the language, things that we call the closed class, like preposition, articles like a, the, those kinds of things.

Annie Duke (00:05:16):
So she was thinking about, well in that case, there isn’t… it doesn’t seem like there’s a very clear mapping between the world and the things that are being said, not in a way that it makes sense that someone would be able to learn this language in the space of two years. And I think that we can all feel that when we’re trying to learn a foreign language, how incredibly hard it is, and we even have a language already that we can base it on.

Annie Duke (00:05:38):
And so kind of taking a lot of influence from Chomsky, what she said is, “But there’s a structure to the language, and the structure is very specific about what a word could or could not mean.” So as an example, if I say to you, the man dacked into the store. Even though you don’t know the word dacked, I have confined, it can’t be believe, right, it can’t be thought, it has to be something that has walked into, stepped into, looked into. I’ve constrained the set of things that could be. And if I gave you more structures within that, if I said the man dacked into the store, but also I could say the dog dacked, right, then that starts to constrain it even further.

Annie Duke (00:06:20):
And then if there were some way that the child had access to the structure of the language, that the structure actually creates real constraints about what any word could or could not mean, right, and that this would allow the mapping to occur more quickly.

Annie Duke (00:06:33):
And so she just started sort of thinking about that problem, and really just brought to many, many brilliant students like Allissa Newport, Suzan Goldmedal, Sharon Armstrong, Barbara Landow, just a variety of really, really brilliant, brilliant students who have come out of her lab, Gary Marcus is another one, started building this body of proof that the child does actually have access to the grammar. And one of the amazing proof points of that was that, if you take children who are deaf, who can’t hear, and you deny them sign language, so you don’t give them any language input, which actually happened in the ’70s, there was a movement to try to make them be able to read lips, and so you were told not to allow your children… not to sign to your children. The children themselves, create a fully grammatical sign language, with no input.

Annie Duke (00:07:24):
So it looks like there’s kind of like a language function that’s built in, where you will produce these fully grammatical languages. There’s a variety of other proof points for this, and that then allows the child to sort of bootstrap what do all of these words mean, and then build out the language. So you kind of think about it as, there’s certain settings for the grammar, is it more like German, is it more like French, because there are some grammatical differences. And once you sort of figure that out, you can unlock the whole system.

Annie Duke (00:07:55):
So she was working right up until the moment that she died. And I went to graduate school, I studied under her, that was really mainly what I was working on, was this concept of syntactic bootstrapping. And she was really, really amazing. And then right at the end of graduate school I got sick as I was going out for my job talks actually, and I ended up taking a year off and then during that year was when I played poker, and I lost touch with her. Which was my fault, not hers. I was just ashamed that I had left graduate school, I felt like I had let her down.

Annie Duke (00:08:22):
I had a big talk track happening for me about what a failure I was, and we reconnected a little over a decade before she died, and just accidentally. I saw her in a doctor’s office, actually, and it was like no time has passed. And it’s one of those amazing moments for me where, I had this whole track about how ashamed she was of me, and it turned out that she was just proud and sad that we hadn’t been seeing each other. And from that moment on, we started basically having lunch literally every week at least. And we just picked right back up where we came from.

Annie Duke (00:08:55):
I found out that she was actually really proud of me, and she had been cheerleading me the whole time, and I got this person back who was this incredibly important figure in my life intellectually, and also emotionally. And I’m very, very grateful for the last 10 years that I had with her.

William Green (00:09:11):
I was looking at an interview with her, and she had this kind of impish charm, and you could just-

Annie Duke (00:09:18):
She is so-

Annie Duke (00:09:18):
… impish.

William Green (00:09:18):
Yeah, just-

Annie Duke (00:09:18):
Just so mischievous.

William Green (00:09:19):
Yeah, so funny and smart.

Annie Duke (00:09:21):
Yes.

William Green (00:09:22):
I showed a picture of her to my daughter over Zoom yesterday, my daughter Madeline that’s at college, and my daughter’s like, “That is one cool woman.” You could just see she exuded attitude, and fun, and-

Annie Duke (00:09:33):
I mean, she had to claw and fight her way into a place in academics at the time. You know there weren’t a lot of women around, and she literally… I mean, she was probably the smartest person in the room, and she had to be a research assistant in order to be able to do her work. And she just was like… I mean, she was just so funny, just one of the funniest people ever. And the thing I can say about her is, as much as I loved her, she really loved me and I’m really sad that we lost that time, which was because of my own feelings of shame about whatever, not having fulfilled the things that I thought that people were expecting of me.

Annie Duke (00:10:11):
And I’m so grateful for that day I was sitting in a waiting room and there she was. And I went over to her, and it was… And this is one of the things about her, I mean, I think that there are people who might have been very irked to say the least, that we had fallen out of touch after putting so much time into my education. Our relationship was quite close, and I think a lot of people could have really held that against me. And I walked over and the minute she saw me, it was just this huge smile on her face, and there was never one minute, not one.

Annie Duke (00:10:43):
And when I visited her in the hospital a couple of days before she died, we had talked about it a little bit before, but when we were in the hospital I actually told her, I just said, “You know Lila, I’ve lived with so much shame over that for 20 years, and I’m so sad. I’m just so sad that I missed that time, and I’m so afraid that I disappointed you and that it wasn’t okay.” And she just said, “Not for a second should you ever think that. I have never done anything but love you, and I’ve never been anything but proud of you.” And I mean, she was just that type of person, and I miss her every day, I really do. And I hope she would be really proud of what I’m doing now.

Annie Duke (00:11:20):
I know… Here’s the thing, we were talking about… She was really encouraging me to go back and finish my dissertation, I’ve been talking about it for a little while, and she knew that the plans were afoot at the time that she passed. And as much as I really hope she’s proud of this, I really hope she’s proud somewhere to know that I’ve now enrolled back at Penn and I’m finishing up. And on the days where I’m like, “Why am I doing this,” it’s always like, because I can just see the smile on Lila’s face.

William Green (00:11:49):
This is an amazing thing that we need to sort of explain to our listeners, that basically, here you are this kind of famous author whose written three, I think, books, and become kind of a very well-known speaker and expert on decision-making. And here you are in early middle age, going back to college to finish the five years of research that did under Lila, is that fair to say?

Annie Duke (00:12:11):
Yeah. So the rules are, if you’ve been gone for too long, you have to a tiny bit start over, not all the way. So the way that graduate school, at least at Penn, the way the program works is that, the first two years you’re learning, you’re doing seminars, and then… Well, the first two and a half years, and then the second semester of your third year, you would so something called qualifying exams, which would qualify you to be a Ph.D. candidate. So that’s essentially showing a command of the literature that you’re working in, the space that you’re working in, and then after that, that’s when you can start to TA. Well, actually you can… Well, you TA before that, but you can start to teach your own classes because you’re considered qualified. And then you’re putting together your dissertation for the last two years.

Annie Duke (00:12:49):
So I was actually on my way to job talks. What that means is that, I’d already done all of my research for my dissertation, it was ready to go, I just really needed to defend it but I got sick. I got really sick, I ended up in the hospital for a couple of weeks, and I needed to take time off. That’s when I started playing poker, so I didn’t defend. So now-

William Green (00:13:04):
So you-

Annie Duke (00:13:04):
… I’m trying to go… What?

William Green (00:13:06):
To go back to that. So you had what, it was something called gastroparesis, which is some sort of disorder-

William Green (00:13:11):
Sorry, yeah, you can see my knowledge of medicine.

Annie Duke (00:13:14):
Yeah.

William Green (00:13:14):
So it’s some sort of disorder of the stomach, right? So you’re in your final period at-

Annie Duke (00:13:21):
Oh no, I’m literally about to go to NYU for a job talks, to go hopefully get a tenure chart position.

William Green (00:13:27):
And so then as I understand it, so you were a start student, right? You were a National Science Foundation fellow and the like, you were applying for jobs to become a college-

Annie Duke (00:13:36):
Oh, I was [crosstalk]-

William Green (00:13:36):
… professor. But you thought you were going to become a college professor, and then suddenly at the age of 26 you get sick, and it catapults you from one path into this whole new direction.

Annie Duke (00:13:47):
Yeah.

William Green (00:13:47):
Can you talk more about becoming a poker player and the sense of shame, because this was not a… I mean, it was a slightly tawdry and disreputable profession in those days, right? If anyone even considered it-

Annie Duke (00:13:59):
It was, yeah.

William Green (00:13:59):
… a profession. So can you talk about that shift, and the role also that your brother played in shifting the direction of your life away from this incipient academic-

Annie Duke (00:14:11):
Absolutely.

William Green (00:14:11):
… career, into a raffish poker career?

Annie Duke (00:14:15):
Sure. So let me just say also about re-enrolling is, they’re not making me do the whole thing over again, so I’m writing the dissertation, as soon as I defend that I’ll be done. And I’m doing that with Phil Tallack, I don’t know if you’re familiar with his work.

William Green (00:14:26):
Yeah.

Annie Duke (00:14:26):
He’s fantastic, he’s amazing.

William Green (00:14:26):
Yeah, he wrote a great book.

Annie Duke (00:14:27):
And his wife, Barmella, who is equally amazing.

William Green (00:14:30):
We should talk about Super Forecasting later, because I know that-

Annie Duke (00:14:33):
I’d love to, because that’s what my dissertation is on. We’re actually working with Novice Forecasters, which is super fun-

William Green (00:14:38):
Yeah.

Annie Duke (00:14:38):
… trying to help them to become better forecasters-

William Green (00:14:40):
It will make no difference, Annie, because the expert forecasters are equally lousy. So the novices-

Annie Duke (00:14:46):
Well, yeah, it turns out you can turn people who’ve never forecasted into really good forecasters pretty quickly. But we’ll talk about that with-

William Green (00:14:53):
Yeah.

Annie Duke (00:14:54):
… handing them the right concepts.

William Green (00:14:56):
It’s funny, because I mean, Tallack writes that basically the experts do less well than chimpanzees throwing darts, and so yeah, I-

Annie Duke (00:15:04):
Well, except for the super forecasters-

William Green (00:15:05):
Yeah.

Annie Duke (00:15:06):
… who do better than CIA analysts, and so-

William Green (00:15:15):
Yeah, so let’s go back-

Annie Duke (00:15:15):
So with the poker playing-

William Green (00:15:15):
Yeah, the-

Annie Duke (00:15:15):
Yeah, the poker thing.

William Green (00:15:15):
… dramatic shift at the age of 26.

Annie Duke (00:15:15):
So I get sick, I actually had job talks lined up at a whole bunch of places, I think it was NYU, Duke, the University of Oregon, I think Cornell-

William Green (00:15:23):
Yeah, Texas.

Annie Duke (00:15:24):
… UT Austin.

William Green (00:15:25):
Yeah, yeah.

Annie Duke (00:15:25):
So I was really on the tenure track path and I get sick, I end up in the hospital for a couple of weeks. But I want to just explain to everybody, that it’s 1992, I think, something like that, 1992. So it’s hard to imagine, because now we see poker on the television all the time, it seems like… I mean, it’s an odd thing to do, but certainly people know that you can do that for a living. And they think that it’s a relatively respectable thing to do now, I think.

Annie Duke (00:15:52):
But at this time, there was not poker on television, obviously there wasn’t internet poker, so this was not anything that was in the realm of possibility for what somebody consider doing for a profession. But I needed money. So I didn’t have my fellowship, because I wasn’t in school, I was taking time off and I needed money. So my brother had gone to New York in the ’80s in order to become a chess player, he wanted to be a grand master. So he was a master, but he wanted to be a grand master. So he was studying with someone there, taking a year… deferred a year of college in order to do that, and in that time he started playing poker, he actually became really good at it. So there was like a whole underground poker scene in New York, he started winning a lot of money at that, actually made it to the final table of the world series of poker at the age of 23. And so that was all happening sort of at the end of high school for me, and then also when I was in college in New York for four years and I would sometimes go watch him play, certainly when I was in graduate school. And during graduate school, he would bring me out to watch him at the world series of poker, but it was boring, and so I asked him if I could play.

Annie Duke (00:16:56):
So I played a little recreationally, but that was it. So now all of a sudden I’m like, “Oh no, I don’t have any money,” and he suggested that I play poker in the meantime to make some money. And I understood enough about the game, I was better than the average bear just because there was somebody who was a world champion basically who was related to me. And he helped me to sort of work through the game, and I started playing and I started winning right away. And it was kind of the perfect thing for me to do at the time, because I didn’t know day-to-day how I was going to feel, and so I needed something where I was like, “Make your own hours,” which poker definitely is, where you weren’t obligated to a boss. There was no calling in sick or something like that, and I did really well. But just so that people understand, at that time, again, it’s not on television. I mean, people definitely think about this as something that the vice squad should be dealing with. So when I would tell people that I was playing poker, it would be kind of like, “Are you dealing drugs on the side also?” But I think because I was a woman, a lot of times it would be, “What does your husband do,” because I think they assumed I had a gambling addiction, and so my husband must be really rich.

Annie Duke (00:18:03):
I think it’s interesting, my husband was a house husband at the time by the way, so I was actually making the money. But it was always just weird, right? And then they would often ask me if I’d been to Gamblers Anonymous, that would also be a thing that they would ask. So it wasn’t… And I tried to explain to people, it’s like investing, but I don’t think… Now I think that people sort of realize that, right, but at the time they didn’t.

William Green (00:18:24):
And in a way, Annie, it seems to me that just looking back on your earlier book, Thinking in Bets, and on this new book, Quit, that it was this amazing education for you in terms of learning to respect uncertainty, which is obviously one of the great themes both in poker, but also in investing. And I wonder if you could talk a bit about that, about how you came to understand that it was kind of this perfect microcosm of life, that you start to see just how complex and uncertain life and games like poker and investing are?

Annie Duke (00:18:56):
I think what happened was that the first… I really started playing for real in like ’94, and I would say that the first eight years, I was just trying to figure the game out, which is trying to sort of get your arms around the uncertainty that, really understanding that you have to get down to what do I have control over, and what don’t I have control over. And I have to accept the tremendous influence of luck, I have to accept the fact that I’m having to make these very high-stakes decisions without being able to see my opponents cards. That’s just the nature of the game is that, you don’t have very much information, you’re having to build models and try to make these maps of mapping your behavior, William, onto the cards that you might be holding, right?

Annie Duke (00:19:44):
And it’s very complex, right? And then there’s also the issue of just applying base rates to that problem, right? Understanding how often do you get certain hands, how often does someone generally enter a pot. If I understand that you’re going to enter the pot 25% of the time, then that tells me something about likely what you have, are you above the base rate, are you below the base rate, those kinds of questions. So you start to get some good anchor points for making these forecasts, but then just be really accepting of the fact that, “I can get my money in the pot with 98% advantage, and 2% of the time that means I’m going to lose.”

Annie Duke (00:20:19):
And you just have to be okay with that, because you don’t have control over that, you’re going to observe that 2% of the time, and that you have to not let that mess with you. Which is actually quite a hard problem. So I was really trying to sort of tackle that for myself. And then in 2002, this was right when poker was sort of coming on TV, in 2002 I got asked by someone named Roger Lowe, who at that time he had founded a hedge fund called Parallax, and he wanted me… Well, he actually wanted Eric Cidell to come and speak to his traders, but Eric Cidell couldn’t do it for whatever reason, so he asked me to do it.

Annie Duke (00:20:49):
I was two weeks from having my fourth child, so I was very pregnant, none of my shoes fit so I actually did not wear shoes when I gave this talk to these auction traders. And that was the first moment where I really thought in an explicit way, about the way that my background in cognitive science, and sort of really thinking about it… I remember, I was thinking about these mapping and uncertain systems, right, that’s where I was thinking about it, and also I’d done a lot of work in just sort of general cognitive psychology, bias, I’d taken seminars from John Barron, whose one of the giants in that field, thinking about the way that poker and cognitive science could have this very interesting conversation that then applies to any type of decision making that you might be thinking about.

Annie Duke (00:21:33):
And so he had… Roger had asked me to speak specifically about how poker informs your thinking about risk, and what I ended up doing was actually saying uncertainty distorts your risk attitudes. And particularly, the way that we sort of first of all process wins and losses, in terms of how do we assign that, right? Do we assign that to skill elements, or luck elements, which is really hard to do on retrospect, right, because you don’t know for sure. And then how does the path that we’re on distort our risk attitudes going forward?

Annie Duke (00:22:03):
So this was very much like a collision of my background in cognitive psychology, and the things that I had been thinking about in poker, which I think you can see expressed in… I mean, I think that was the first time that I started writing Thinking in Bets, because that is what Thinking in Bets ends up being about, is that particular problem in large part. And then I just realized, “Oh, I really like this conversation,” and I just kept going at it. And that was just sort of the start to really thinking about this explicitly.

William Green (00:22:31):
It’s interesting how so often in life, it’s these unexpected things that don’t seem to have any relationship to each other, that come together and somehow create something new and fresh, that it was your weird interest in cognitive bootstrapping, and the acquisition of language, and whether there’s some grammar innate in the mind when we’re born or something, and then that somehow leads you into poker kind of randomly. And then you sort of, you understand poker. And I think also Lila had kind of taught you to think very scientifically, in a way that very few poker players would too.

William Green (00:23:09):
So you’re sort of bringing together these different strands, and then you stumble into the financial world and so you’re able to apply… Does that make any sense, there’s something kind of weird and beautiful, and sort of random and sort of lucky, and sort of fated?

Annie Duke (00:23:22):
Yeah, I mean, there’s definitely a lot of luck, and there’s definitely a lot of randomness. I mean, to harken back to David Epstein, there’s a lot of range.

William Green (00:23:33):
Yeah.

Annie Duke (00:23:33):
And I think that one of the things that… I think when people look at my life, I mean, I think they tend to describe it as you just did, which is, “Oh, what a lot of random things that you were doing.” But when I look back at it, I say, “I have never done anything which wasn’t pulling the exact same thread.” And that thread is, how are you learning, how are you making decisions, how are you closing feedback loops under uncertainty? Because that’s the whole language problem, right, is closing the feedback loop in this incredibly uncertain system where it’s just not clear.

Annie Duke (00:24:05):
It’s like I point at a dog, am I talking about panting, barking, fur, a paw, the dot itself, an animal, a mammal, right? Thinking, sleeping, what on earth does that mean and how does this little being actually manage to close those feedback loops, right? And it has to do with constraints in the system, that then becomes a big part of the way that I think about poker, which is, how do you constrain the way that you’re processing that world, in order to avoid the mistakes that you might make in mapping an outcome to, why did that outcome occur?

Annie Duke (00:24:48):
Which I think is such a huge problem for us as decision-makers, right? You lose on an investment, was it just bad luck? Was your thesis wrong? Was some part of your thesis wrong? What was the contribution of either one of those two things? And I think it’s incredibly hard, and left to our own devices I think we do a very bad job of it. And so starting to think about those constraints, which is really the solution within the language base and how do we take that type of solution and apply it to our own decision making, to both say, which is this wheel in the end, it’s embracing the uncertainty. It’s saying, “I’m not going to try to get to 100% certain, because I think that’s ridiculous to try to do that.”

Annie Duke (00:25:26):
We can’t do that, and if that’s our goal, we’re going to be really bad decision-makers because we’re not acknowledging the position that we’re in as decision-makers. So instead, what I have to do is embrace the uncertainty, and also embrace the mind-ware that I came in with, right? Like I just… There are biases that are innate to me, that’s separate and apart from the noise in the system, right? The noise is the embracing of the uncertainty, and then I know that I’m going to be viewing the world in a biased way that’s going to be relatively systematic. And so how can I create constraint, that will then allow me to be a good decision-maker within the environment that I have to make decisions? And I don’t think that there is anything in my adult life, that I’ve ever… I don’t think I’ve thought about anything else. And I know it’s graduate school, and language acquisition, how on earth could that possibly relate to poker, and how on earth could that then relate to the books that I’ve written, or the consulting that I’ve done, or helping a company that’s a SaaS startup to work through the stuff that they’re doing, and whoa, isn’t that a lot of different things? But no, it’s the exact same thing.

Annie Duke (00:26:27):
I do the exact same thing, I just have been lucky enough because of weird stuff that’s happened in my life, to have been able to think about that thing as it applies to what look like very different problems, that I think have really informed that in a good way. So in the end, I kind of look back and say, “Wow, that was really lucky that I got sick.” It wasn’t lucky that I lost two decades with Lila, that’s my fault. And had that not happened, I would not have reacted that way and done that. But in terms of everything else, it was incredibly lucky.

William Green (00:27:00):
Yeah, I remember Ed Thwarpe once when I asked him how you-

William Green (00:27:03):
Yeah, I remember Ed Thorp once when I asked him how you approach life as a game. He said, well look, there’s the stuff like your DNA that’s just the hand that you are dealt, and then there are the decisions you make about how to play it, like whether you get vaccinated, whether you have your annual checkup, whether you exercise, whether you eat well. It sounds so logical and obvious once you hear it, but it’s not that obvious.

Annie Duke (00:27:24):
Yeah, but then we don’t. We’re so bad at it.

William Green (00:27:26):
Yeah. So I wanted to ask you actually about someone whose name you mentioned, which in your world of poker, everyone will know, but for many of our listeners, he won’t be familiar, which is Eric Seidel. So because your brother had this extraordinary group of poker players around him in New York when you were coming up in the game-

Annie Duke (00:27:45):
I met Eric Seidel when I was 16 years old.

William Green (00:27:47):
Amazing. So for people who don’t know, Seidel… and I know so little about poker, I can’t even tell you, I’m embarrassed to admit, but he, as I understand it, won nine world series of poker bracelets and something like-

Annie Duke (00:28:00):
He’s ridiculous.

William Green (00:28:01):
Yeah, he made something like $40 million as a player so far, and was also a stock exchange trader before he became a professional poker player. So you’ve written before that he was in some ways the person who taught you what it means to strive to be a rational thinker. And you’ve also said that you have a crush on his intellect. So I wondered if you could explain to us what you learned in this sort of unlikely field about how to think more rationally, what did Seidel teach you?

Annie Duke (00:28:32):
So Eric Seidel is an amazing poker player, really incredible. And one of the things I think that people need to realize that’s so amazing about him is, first of all, there’s a lot of short-term luck in poker. So there are people who come and they have a great year, and then they just never do well after that, for example. And partly, I think because people figure them out and they don’t have a strategy to figure out them out… back out, I guess. And you see that in a lot of sports. Someone comes in and does something really unusual. At first they do really well, and then their opponents figure them out. And if they don’t have a response, they’re not going to do particularly well.

Annie Duke (00:29:14):
But also, the game itself has changed. So there’s lots of people who will have a great year and then it turns out they just got lucky. But even if they didn’t get lucky and they really were great for that time, as the game changes, as new people come in, are they able to adjust and do well in those new environments? And that’s something that we’ve seen particularly with the introduction of some deep learning. The strategy that used to be considered correct in backgammon for example, or chess, what we’re finding is that AIs are telling you to do very different things, generally to be more aggressive. And then as the new generation comes in and they’ve sort of metabolized those lessons, are you, as someone who’s been doing this for a very long time, able to adjust back and change your game in response, which is very difficult to do? so here you have Eric Seidel who’s been doing this now for four decades. And it just year after year, after year, one of the top poker players.

William Green (00:30:13):
And I remember you telling a story about going to him and complaining about some game. I think the term you used was “a bad beat” where you just sort of got unlucky. And how did he respond when you complained about your bad luck?

Annie Duke (00:30:28):
Yeah, so this is one of the things that I think makes him so great, which is… poker’s a really tough environment, because there is a lot of luck, and you are going to have situations where you are absolutely a gigantic favor and you’re going to lose. And by the way, sometimes you’re going to think that you got unlucky but you actually didn’t. But that’s a whole… I mean this is the issue of what he was trying to get at.

Annie Duke (00:30:53):
So I came to him. This was one of the first final tables that I ever made. And I remember I was raised with two jacks, and this guy, Gus, who was from Costa Rica, those are the things I remember about him, moved all of his chips in. And this was either, if I win the hand here, I’m going to be the chip leader in the tournament with six people left, and if I lose, I’m going to be out six. So this is a pretty big swing here. And so he moves all his chips in and I thought about it. I really thought about it for a long time, because jacks is sort of a middle-of-the-road hand. And I ended up calling. I really just decided that he didn’t have a great hand, and he had two nines.

Annie Duke (00:31:34):
So just to set the stage I’m going to win that pot around 82% of the time, just shy of that. And he won the hand. And this was the biggest stage I’d ever been on. It was the most money I had ever played for. I mean this was a really big deal. Had I lost hands like that before? Sure. But never when it mattered so much.

Annie Duke (00:31:53):
So I walked away from that table and I was just really upset, and I went to him and I was just complaining to him. And I was like, “Can you believe this? I made such a great call and this idiot moved in with two nines,” and blah blah blah, blah, and this whole thing about how unlucky I was. And he just stopped me and said, “Is there a question here?” And I was taken aback, because you would expect most people to be like, “I am so sorry; that’s so unlucky. I feel so bad for you. It really mattered to you.” I mean I think I was 26 or 27… something. I was young. And that’s normally what you would expect, but he was just like, “Is there a question?” And so it was like, “What?”

Annie Duke (00:32:33):
He goes, “I don’t want to hear about it if there’s not a question. I don’t care that you got unlucky. I get unlucky too. And I have to deal with losing with two jacks against two nines all the time also. I certainly don’t want to take on your emotional trash about it myself. And what’s the point of talking about it? You made a great call and lost, who cares? Would you have changed anything about what you did? Do you think you got the read wrong? It sounds to me like you did everything right. So why are we even talking about this?” I was like, “ugh,” right? And at first, I was really mad, and then I realized, no, he’s totally right. I mean this is the thing, if it really was just bad luck, who cares? This is about embracing that uncertainty, right?

Annie Duke (00:33:18):
If you have a question, and he said that to me, if you have a question, if you think maybe you shouldn’t have called there, or maybe you shouldn’t have opened, or maybe you should have opened for more, maybe you should have moved in then for… maybe you should have done something different, then I will talk to you about that all day. But if all you’re talking about is that the poker gods came down and were mean to you, I don’t care because it doesn’t help you going forward.

William Green (00:33:45):
It’s a beautiful story. There’s great depth in that. I wanted to pick up on one thing that you mentioned there where I think you said there was an 82% probability that you were going to win that hand if I heard correctly.

Annie Duke (00:33:58):
81.5% or something, yeah.

William Green (00:33:59):
81.5%. And I’ve interviewed a lot of great investors over the years, people like Charlie Munger, and Ed Thorp, and Howard Marks, and Bill Miller, Joe Greenback, these guys who just think constantly in terms of probabilities, they’re always looking at the future as a distribution of probabilities, and they’re assigning odds to those different probabilities. And I once asked Howard Marks if it was possible for an investor like me who’s not really wired that way to learn to think probabilistically. And I was kind of disappointed. He was like, “Yeah, no, probably not.” And I was interested in this whole question of A, how important it is to learn to think in terms of probabilities, and B, whether you actually think that is something that we can become much better at.

Annie Duke (00:34:41):
Yeah, so I think I’m going to disagree with Howard Marks here. So I think I’m going to disagree on two counts. And by the way, I love Howard Marks, he’s super, super smart. And actually, he may not disagree with me when I phrase it this way, let me just say that.

William Green (00:34:52):
He may just have meant me, that I could never learn to think probabilistically.

Annie Duke (00:34:56):
No, but I think… no, because Howard and I have actually talked about this, and I think here, he’s going to agree with me because I’m going to reframe it. And I think with the reframe, he’s going to be on the same page as me. Okay, here’s thing number one, you are thinking probabilistically. So I think this is really important to understand. So I actually had someone recently say to me, “Oh, you’re talking about probabilities and expected value and those kinds of things, but that doesn’t apply to marriage, because that’s like a one-time decision where you just got to go with your heart,” or whatever. And my response was, “Oh, do you just walk out onto the street and marry someone at random?” And of course the answer is no. And it’s because you are making a forecast of the future. You are saying, what have I learned from people that I dated? What is my model of myself? What is my model of this person?

Annie Duke (00:35:41):
And yeah, just because you’re not going to repeat the decision over and over again, obviously you’re still doing a forecast. You’re still saying, I think my expected value is higher if I marry this person given the constraints of the time that I have. Maybe your biological clock, so on so forth, what your values are in comparison to continuing to search for other people, right? And the fact that you don’t-

William Green (00:36:01):
And also, even if you… sorry Annie to cut you off, even if you look at the… I had a close relative who was a serial philander who got himself in trouble, and he’s playing the odd. He’s like, “What are the odds that I’m going to get caught and this is going to ruin my family?”

Annie Duke (00:36:17):
Well yes, that would be a good negative example of thinking probabilistically, but exactly right, exactly right. Look at things that I value-

William Green (00:36:21):
And I remember warning him as a kid and saying, “Look-”

Annie Duke (00:36:25):
You’re going to get caught if do it enough times.

William Green (00:36:27):
And he was like, “No, no, no, no, no,” and then he got caught and it ruined his family.

Annie Duke (00:36:31):
Right. And hopefully you learn from that. So that’s the thing is-

William Green (00:36:33):
I hope so.

Annie Duke (00:36:35):
Even if you don’t think you’re doing it explicitly, literally every single decision you make is probabilistic, because it’s a forecast. It’s a forecast made under conditions where you don’t have all the facts. You generally know very little in comparison to all there is to be known. And there’s going to be the influence of luck on the outcome. And that’s true. Look, when you decide a particular route that you’re going to take to work, it’s a forecast. I think this is going to get me to work in the time that it’s going to take me to get there, or maybe you value the scenic route. And so you’re saying this is more likely to let me see lots of scenery in comparison with when I need to be there. And then if there’s an accident on the road and you decide to exit to take a different route, that’s an expected value calculation. You are thinking probabilistically. If I exit, even though normally this would take longer, there’s a higher probability I’m going to get there on time.

Annie Duke (00:37:29):
So I think that’s number one, is that we have to reject the idea that if you’re not doing it explicitly, that you aren’t thinking probabilistically because every decision is a probabilistic decision just by its nature because the world is probabilistic, that is how we decide. So that’s the first thing that I just want to say there.

Annie Duke (00:37:49):
Now the act of trying to make these things explicit will make you better at it, because what it will start to do is allow you to create good feedback loops. Remember this is my obsession. How do we start to create good feedback loops? Well, so I’ll give you an example from a company that I work with that had this implicit/explicit thing. So I’m a special partner focused on decision science at First Round Capital Partners. So there is a seed stage company. So when I first started working with them, there were certain things about their voting process that we decided that we really should change to fit with what we know about the science of decision-making. And one of them was to include some very explicit forecasts in what they were in their decision. So one of the forecasts that we wanted to include was, what’s the probability that the company that you’re currently considering is going to successfully raise a Series A?

Annie Duke (00:38:44):
Now the pushback that I got from the partners was, “Well, we don’t even think about that because what we care about is it going to exit for $1 billion dollars?” And so I rolled it back again, this isn’t implicit in the decision, and should we make it explicit, because I felt it was implicit in the decision. And the way that I got to that was to say to them, “Well, once you funded a company at seed, have you ever had a company that was a fund returner that didn’t successfully funded its Series A?” And they said, “No, we have not.” And I said, then it’s in the decision. It’s implied that one of the predictions that you’re making when you decide to invest is that this company will successfully, above a certain rate fund its Series A. So we now made that an explicit part of the decision process, so they have to make that forecast.

Annie Duke (00:39:32):
And now what that does is it gets them to think about things that are actually part of the decision in a probabilistic way, and we can now close those feedback loops, because you do that enough times, you can start to see, well how good are a predictor, are you better than random, knowing which company when it comes in the door is going to fund at the next round or the round after that. And we’ve actually started to be able to close those feedback loops. They are better than random. They are quite good at it. It maps to experience. The more experienced you are, the better you are at predicting that. And that does great things for the junior partners because they see that the senior partners are better at this prediction. They start to try to understand their rationale for that, which then improves their ability to forecast those things. And what happens is they get better.

Annie Duke (00:40:15):
Now, are they perfect at it? Of course not. They’re not perfect at it, but they’re better than if you just threw darts. And that is, as you know, this is where we start to get those edges that we can grind over and over and over again in investing. So I just don’t accept it. I say you are thinking probabilistically, so let’s start making it explicit. And if you start to do that explicitly, you will get better. Will you be perfect? No. Do some people think this way more naturally than other people in an explicit way? Sure. But every little bit that you improve is going to really matter to you for your outcomes.

William Green (00:40:54):
How transferable have you found the skills that you developed in poker when it comes to the investing world? How much is it the same thing and how much is it actually a different type of game?

Annie Duke (00:41:06):
Yeah, so this actually gets into a little bit of what my doctoral research is about. So there’s a whole history of question about the transfer of training from one domain to another, and the history is pretty dismal. It actually starts way back in the early 1900s, when the idea around education was like if you if do hard math problems, like trigonometry, it will help you to be a better thinker in all other domains, and it turns out that actually isn’t true. But there’s some work that begins with Fang & Nisbett on a very specific thing that does transfer, which is statistical concepts. So something like the Law of Large Numbers as an example.

Annie Duke (00:41:44):
So if I teach you about the Law of Large Numbers as applied to a particular domain, I can get very far transfer, where you will now sort of have this concept in your head, you’ll understand that large numbers matter, and it will transfer into other domains. So that’s the thing that I think was so helpful for me in poker was there’s a whole bunch of statistical concepts, which also I had learned in cognitive psychology because you’re having to work with data all the time. And so, you’re thinking in that way already. And then in poker, you have to start to think about things like base rates, for example. And a base rate is just how often does something happen in a situation similar to the one that I’m currently considering?

Annie Duke (00:42:22):
So that’s where I say the average player will play 25% of their two card combinations and hold them. So, that helps me with a starting point. And that concept of base rates actually transfers very well, the Law of Large Numbers, which becomes very important in poker; I don’t want to think about what happened on a particular hand, I need enough reps to be able to say something about it. Those types of things, the way that you view the world through these kinds of statistical mental models and statistical concepts transfers really, really well. So I think that that’s actually been incredibly helpful to me, because everything looks a little poker-like when you start to think that way.

Annie Duke (00:42:57):
And then the other thing is that I just think that depending on the type of investing, it’s actually relatively near transfer. So when we think about near transfer and far transfer, which is just how easily can you map one domain onto another, and if you take something like poker and options trading, now you’re talking about the relatively near transfer. They’re very, very, very similar to each other. So in fact, as I’m sure you’re aware, there are large options trading firms like Susquehanna International Group that actually force their traders to play poker, because those are so similar.

Annie Duke (00:43:32):
Now obviously when you get into something like seed-stage venture capital, you’re starting to get into a much farther domain, but these statistical concepts still apply. And so I have found it to be incredibly helpful. It’s just a matter of not thinking about it as an analogical transfer. You’re trying to take analogies from poker and lay them onto other areas, which I don’t think transfers particularly well. That doesn’t work very well. But thinking more about what are the statistical lessons and underpinnings and concepts that you would then transfer to anything that you’re thinking about.

William Green (00:44:06):
It’s really striking to me how many of the great investors have been very serious game players. And I’m thinking, I remember Sir John Templeton told me once that he put himself through college during the Great Depression with his winnings from poker.

Annie Duke (00:44:19):
I mean Ed Thorp, as you just mentioned, right?

William Green (00:44:19):
Yeah, a brilliant gambler who figured out how to beat the casino at blackjack and then baccarat, and even roulette, which is crazy.

Annie Duke (00:44:29):
Yes.

William Green (00:44:29):
Buffet and Munger are passionate about a bridge. I remember Mario Gabelli once telling me that when he was 11 years old, he was a caddy at a golf club and he made money beating people at cards, because they assumed an 11-year-old wouldn’t know what he was doing. And actually, Peter Lynch once told me over 20 years ago, I said to him, “What books should we be reading about investing?” And he said to me, “Well actually, learning to play poker or bridge is way more helpful because it teaches you to deal with probabilities.”

Annie Duke (00:44:57):
Because you have to feel it. That’s the thing-

William Green (00:45:00):
Say more about that.

Annie Duke (00:45:02):
So it’s one thing to talk about if you have a 60/40 shot, that 40% of the time you’re going to observe a bad outcome. And to understand that conceptually is one thing, to experience it is another. I think that this is one of the things that we need to understand is that there’s a whole bunch of things that we think we know because we understand them from a… if I borrow from Daniel Kahneman, their system one and system two, there’s the more deliberative part of our brain, that’s kind of more what we think about thinking about things. And then there’s most of the rest of our thinking, which is just automatic or reflexive.

Annie Duke (00:45:41):
And so there’s one thing for me to conceptually get that 40% of the time I’m going to observe a bad outcome. But if I can grind a dollar, it’s a 60/40, I’m obviously going to make a ton of money. I’ll take 3:2 on an even dollar-to-dollar every single time that I can. So I can know that intellectually, but when you experience betting that dollar even money where you’re a 3:2 favor and you lose, that is a whole different animal. It really is. And you can see with amateur players and in investors all the time… where they say, well, any two cards can win, but not enough of the time. They forget that part. They’ll have some sort of hand that they fold, and then the rest of the cards will come out and they would’ve won. And it’s so incredibly painful for them that they’ll start playing hands till the end, because that worry about, “But what if this is the one that wins,” it’s so awful for them to experience that situation that they won’t let go of the hand because of it.

Annie Duke (00:46:42):
Because of that feeling of when you actually lose, it’s so hard. So when you play poker a lot, you just become very sanguine about that stuff. You just sort of learn, well, it was a 60/40 favor. What was I going to do? Kind of in the same sense of what Eric… that’s what Eric was trying to get across to me, you are going to win the hand 81.5% of the time. What are you talking about? Go play it again. Would you call again? Would you play the hand exactly the same way and run it 100 times? Of course. So why are you worried about this one time? And that’s the thing that I think that games playing gets for you, separate and apart from just starting to understand this strategy and really starting to figure out how do I model people in these uncertain systems. Poker’s one long game. And if you don’t have the reps, you’re never going to be able to get that under your belt and try to find a way to succeed in that environment. I think it’s just really hard.

William Green (00:47:36):
I remember also Ed Thorp saying to me, “As far as gambling is concerned, if I don’t have an edge, I don’t play.” And I said to him at one point, “Well, so how can I tell if I have an edge when it comes to investing”-

Annie Duke (00:47:48):
And that’s the whole thing.

William Green (00:47:50):
Yeah, and he looks at me and he says, “Unless you have a rational reason to believe you have an edge, then you probably don’t.” And you just sort of hear that and you’re like, “Oh god.” And so there’s that sort of discipline as well that I really learned from Ed Thorp. I don’t apply it myself, but I learned it, which is simply not to play games that you’re not equipped to win.

Annie Duke (00:48:09):
I would actually amend what Ed Thorp said about unless you have a rational reason to win, that you have an edge. So first of all, I think we’re very good at fooling ourselves into believing that we have a rational reason, that we have an edge. And I think that that’s particularly so when we’re in a situation where the thesis would affirm other things that we already believe about the world. I think it’s particularly so when we’re already in the investment. So one of the things that we need to realize is that every day that we wake up and we’re looking at our portfolio is a day that we could sell everything in our portfolio. And so the decision not to sell is saying, “I believe I still have an edge going forward.” But one of the things to understand is that once we’re on a path, particularly if we’re in the losses, if we have a loss on paper, we’re going to do all sorts of ways to convince ourselves we have a rational reason that we have an edge going forward.

Annie Duke (00:49:01):
So one of the things that we need to do is set up structures around us that will allow us, first of all, to be better at those, are we really being rational and starting… but more importantly, because the starting decision is always uncertain, is to say, as we discover new information after we’ve started, are we stopping, right? Are we figuring out when we should stop? Because it turns out that we’re very, very dense when it comes to actually paying attention to the signals after we’ve started something that we ought to stop it. And that’s where we get particularly irrational. And I think that that’s just something really incredibly important to understand.

Annie Duke (00:49:40):
So there’s two pieces there. If you don’t understand the game, don’t play. So I completely agree with that. There’s a reason why I never owned a single coin, because I didn’t understand it. I didn’t want to understand it. At a point in my life where I was like, you know what? I don’t really need to sit down and research Bitcoin and NFTs. I don’t have the energy for it. I don’t want to turn my attention to that. And so I don’t understand it, so I’m not going to play it. So I never invested in it just because I didn’t understand it. But then what we also need to understand is that, let’s say that I was in it, that one of the arguments that I kept hearing as an example for Bitcoin was that it was going to be uncorrelated with inflation.

Annie Duke (00:50:22):
Okay, so look, I don’t know from anything, that seems like a reasonable thesis. We haven’t experienced any inflation, so you probably have a rational reason that you have an edge there because of that piece of the puzzle that it’s going to be uncorrelated, so it’ll act somewhat as a hedge against inflation. And who am I to argue with that? And by the way, who are you to argue with that, because that’s a forecast and we haven’t experienced inflation yet, right? But the question is, what do you do when it turns out it’s not uncorrelated? That’s the big question. And William, that’s what I would ask you. Were people selling when they figured out it wasn’t correlated… that it was very much correlated with inflation? And the answer for most people is no. And so that’s where we need to start thinking about… that’s where that, you better have a rational reason and behave rationally to those signals, I think comes into even much more play is post-decision.

William Green (00:51:13):
So you describe in the book, Quit, a very valuable tool which you call kill criteria, which obviously relates to what we are discussing here, where you develop in advance a set of kill criteria for when to cut your losses. Can you talk more about this and how we should do it? Let’s say we’re about to buy a stock, or we’re about to buy a fund, or we’re about to buy a cryptocurrency. Can you take us through the process of laying out kill criteria, making what you call a pre-commitment contract so that you actually know what you’re going to do when things start to go horribly wrong?

Annie Duke (00:51:51):
So here’s what we need to realize about these decisions about stopping. I think that we, particularly for investors, we get lots and lots of focus on the starting question. What are you buying? What thesis are you deciding that you actually want to trade? What positions do you want to put on? But we need to get a lot more focus on the selling side. And I think that one of the reasons why we don’t focus on it so much is that we have an intuition that if we have a particular thesis that implies certain things about, say, the fundamentals. What are the fundamentals going to look like in the future. That when those fundamentals move against us, that obviously we’re going to take risk off.

Annie Duke (00:52:25):
And I think that that’s true, whether it’s investing or anything else, we think if we run a marathon and we break our ankle, we’re not going to run anymore, or our leg. If we’re going up a mountain and a snowstorm comes in, or the fog rolls in, we’re going to turn around. If we have a business and things just start going to crap, that obviously we’re going to do something about it. And the fact is that just these decades and decades, and decades of research shows that we don’t. We’re not very good at paying attention to those signals when they occur. So with the Bitcoin example, if you’re investing in that, let’s say in part, or maybe in whole, let’s say the main part of your thesis is I’m using this as a hedge against inflation because it’s not going to be correlated with inflation. We think that-

William Green (00:53:07):
Or financial chaos in general.

Annie Duke (00:53:07):
Yes.

William Green (00:53:09):
If the financial system starts to melt down and stocks start to melt down, it’s not going to be correlated with stocks, it’s going to be a hedge. And then suddenly everything falls apart and you’re like, “Oh no, this is falling too.”

Annie Duke (00:53:19):
This is falling too. I think that because that’s part of our thesis, we assume that just because it’s part of our thesis that when those things occur in the future, that would be in a signal that maybe the thesis wasn’t on point, that we’ll react to those in a rational way, which would be to take the risk off. That is what our intuition is. But again, decades and decades and decades of science tells us that that is not what we do. As you say, when financial chaos occurs and things are melting down, Bitcoin melts down too. Do people then now no longer become Bitcoin believers? That’s the question. And I think that we’ve seen, no, that’s not what happens at all. I think it’s a good example of it. Okay, so the question is then, so how do we actually solve for this problem, right? Because if we know-

Annie Duke (00:54:03):
And so how do we actually solve for this problem? Because if we know that we’re not going to be good at paying attention to the signals when they happen, which is so bad because the option to quit is so incredibly valuable. Because when you learn new information after you’ve started something, the option to quit is what lets you do something about it. Imagine if you didn’t have that option. Imagine if when you bought something, you had to hold it for the rest of your life. Oh my God, nightmare.

Annie Duke (00:54:24):
What we need to do then is, say, if we can’t rely on ourselves to react in a rational way to the signals when they occur, what we need to do then, is do that work in advance. This is kill criteria. I’ve done this work with PMs, and basically what we do is we say, “Look, it’s not enough for you to write down what your thesis is, which is work that you need to do. You have to do that quant work. You have to get the analyst, the quant work, model it, make sure we know what your thesis is, and that’s fine, but it’s not enough. What you have to then do is say, “What could I see in the future that would make me say that this isn’t… that my thesis was wrong?”

Annie Duke (00:55:12):
So for example, if you have some idea about correlation and then you see the world unfold and it turns out that it is correlated, it’s not uncorrelated, that would be a good example of a kill criteria. What I could do is I could say, “I could have done this,” because I didn’t really understand Bitcoin. What I could have done in order to deal with the fact that I didn’t know much about it was to say, “I’m going to buy Bitcoin. But the reason I’m buying it is because people say that it’s a good hedge against inflation or general financial chaos. So as inflation goes up, I’m going to see if Bitcoin is going down, and I would have some parameters to try to make sure it wasn’t just noise.”

Annie Duke (00:55:53):
And I would say, “So if this happens, if I see that it’s going down in a way that’s very much correlated at one with what’s happening with inflation, this is how long I will tolerate that situation for.” So maybe I would tolerate it for one point or one and a half points of inflation or something like that, and then I’ll sell.And so I write down in advance what I’m going to do.

Annie Duke (00:56:13):
Now ,I think that people really think it’s a distinction without a difference, because it’s clearly, if you have a thesis, those moves against you are implied in the thesis so why wouldn’t you pay attention to them? And you just got to trust me on this one, you’re not going to. So write it down in advance and it’s what I call kill criteria.

William Green (00:56:31):
Well, I think one of the really important insights is that when you’re actually in the heat of the moment, your emotions go haywire and your rationality just isn’t great. And one of the things I loved when you wrote about poker is talking about different states in which you saw people making bad decisions, whether they were drunk, or tired, or whatever. And you said at one point you knew that, I can’t remember which of your books this was in, that you knew that you played best in six to eight-hour sessions. And so you had to set out the rules beforehand rather than waiting until, as Daniel Kahneman would say, until you are in it. And I think that’s actually such an important insight that we need to recognize before we are in the muck, in the extreme situation that we’re-

Annie Duke (00:57:14):
Yeah, because who’s making good decisions right now? The world is melting down.

William Green (00:57:21):
My friend Ken Shubin Stein, who I write about in Richer, Wiser, Happier, is very, very smart former hedge fund manager and private equity manager who’s now become a neurologist. So he is an expert on the brain.

Annie Duke (00:57:33):
Oh, I love that.

William Green (00:57:34):
Yeah, he’s an amazing guy. And one of the things that he developed, because he studied so much scientific literature, including addiction literature, is he said, Okay, so I know that I’m going to behave sub-optimally when I’m hungry, angry, lonely, tired, in pain, stressed. So he has this mnemonic called PS to remind him of those states. And he said to me, I wrote about this a bit in the book, but he said to me in the early days of COVID, when he was treating patients on ventilators, he could see for example, when he was really stressed or when he was scared. He had a four-day-old newborn child and he moved into a hotel room to keep his child and his wife safe. And he said his equipment hurt, and his back hurt. He had a back injury. And so he had to just be doubly careful because he knew in advance that those were states in which he was going to behave sub-optimally,

Annie Duke (00:58:25):
Right. Yeah.

William Green (00:58:29):
So that-

Annie Duke (00:58:31):
Exactly. And then put on top of it, there are particular states where you’re going to feel suboptimal that have to do with your own physical state. You’re tired, you’re stressed, whatever, but then there are also cognitive states where you’re going to behave sub-optimally. And the particular cognitive state where you’re really going to behave suboptimally is when you’re in the losses.

Annie Duke (00:58:49):
So the reason why I’m calling that a cognitive state, is that it’s not objectively true. So being in the losses, which means that you have some sort of loss on a mental account, it lines up somewhat with what your balance sheet really looks like, but not exactly. So it’s true that if you bought a stock at 50 and is trading at 40, you’re both cognitively in the losses and you’re in the losses in terms of your balance sheet, you’re carrying an unrealized loss. But what’s interesting is if you bought a stock at 50 that went to a hundred and is now trading at 75, you are in the gains in terms of your balance sheet, but you’re in the losses cognitively, because you were at a hundred, you’re now at 75. So you’re experiencing that in the losses.

Annie Duke (00:59:32):
Another example would be if you’re running a marathon and you’ve run 12 miles of the marathon, you’re technically in the gains in the sense of you’ve run 12 miles more than zero, but you’re cognitively in the losses because you’re short the finish line by 14 miles plus. So that’s in the losses. So this in particular, the 68 hours was dealing with the physical issue, but then I also had a loss limit to deal with the cognitive issue, which is that when you’re in the losses, you’re going to be a terrible decision-maker.

Annie Duke (01:00:06):
And the reason that you’re going to be a terrible decision-maker is, except for the part it’s going to cause you to be emotional, is that you’re going to want to get your money back. And this is a really big problem for investors. You start down a path, it starts to lose and you don’t want to sell, because you can’t get your money back. That’s the moment that you go from a loss on paper to a sure loss. It’s when it becomes a realized loss. And that is a moment that we do not like. And so we will come up with all sorts of reasons to think we’re being rational in continuing on, when we’re being completely irrational because we’re just trying to protect ourselves from having that moment of having to take the sure loss. This is what Daniel Kahneman calls sure loss aversion. And it’s related to obviously to some cost. And we better set up some things in advance that stop us from doing that.

Annie Duke (01:01:00):
Now again, in poker, there were things I had, like if bad players got up and good players sat down instead. That would have something a little more to do with the fundamentals. But I also just had simple loss limits in place just because I recognized I’m probably going to be a bad judge of who’s playing well and who’s not. When I’m personally losing, I’m going to be terrible at that. And those were things that I just thought about in advance to try to deal with these issues.

William Green (01:01:25):
So like many people, I’m nursing some losses on individual stocks at the moment, and the most comically awful of them is Alibaba, this Chinese company, which I looked at this morning and I’m down 59% in, I think, probably a year or something. And I bought this after having this amazing Zoom call with Charlie Munger and Lou Simpson and various other great investors. And so, you feel really special. You’re like, Okay, so wait, I’m talking to some of the greatest investors of all time about why they love Alibaba, why it’s so cheap. And so I suffer from authority bias. So I sort of fall in love with the ideas of these brilliant people I talk to and I have access to brilliant minds. And so then I’m like, Wow, I feel so special. I’ve got this insight into what they’re doing. And so then I buy this stuff occasionally, I very rarely buy an individual stock, but I own about three of them, and then it’s down 59%. And so I’m just sitting there watching it, in a somewhat paralyzed, somewhat amused, somewhat fascinated way.

William Green (01:02:29):
And I wonder if you could just talk about some of the cognitive biases that are kicking in here. So there’s loss aversion or sure loss aversion that you just talked about. Talk a bit more about sunk cost effect and status quo bias and things like that, because it’s kind of this Lollapalooza of idiotic cognitive biases that are making it impossible for me to make a good decision.

Annie Duke (01:02:53):
I think people are familiar with loss aversion. We don’t like to start things that carry with them a chance of loss, even if we’re winning to the decision. So we’ll prefer some low volatility, but not particularly valuable, investment over one that’s higher volatility but more valuable, because the higher volatility investment obviously carries with it a higher chance that you’ll lose, and probably a larger possible loss. So that’s a problem with starting.

Annie Duke (01:03:17):
Sure loss aversion on the other hand, is when you already have a loss on the books, or even a cognitive loss on the books, it was trading at one level and now it’s trading lower, we don’t like to sell. In other words, this becomes loss aversion stops you from starting. Sure loss stops you from stopping. In other words, we won’t sell because we already have the loss on the books. So sure loss aversion is related, it plays with sunk cost effect, which is simply that we take into account what we’ve already spent on an endeavor, time, attention, money, on whether we should continue and spend more. In part because we fear that if we don’t, we’ll have wasted what we already spent and we don’t want to do that. And also in part, because we feel like we want to get our money back basically, or our time back. And if we exit, that’s the moment that we can no longer do that, but as long as we continue, we could recover the cost.

Annie Duke (01:04:16):
Of course this is irrational, because what matters is, is the next dollar that you spend worthwhile, not did you already spend a dollar. We shouldn’t care. So if you’re holding a stock at 40, that’s the same thing. Less transaction costs and saying I’m willing to buy this stock at 40. But what happens is that, even stocks that you wouldn’t buy at 40, today you’ll hold because you’ve already lost on the position to date and you’re trying to get that back. That’s an error in the way that we do mental accounting, because we don’t think about it as fungible across a portfolio. So that’s the sunk cost effect that stops us from starting, stops us from stopping rather. Then bring loss aversion again as opposed to sure loss aversion, back into the equation with this stopping us from starting, that loss aversion does, because when we quit it’s usually because we’re considering starting something new.

Annie Duke (01:05:09):
So what ends up happening is that we have this very strong preference for the status quo. Whether it’s the job you’re already in, or an investment that you’re already holding, a relationship that you’re already in, a project you’re already doing, a product you’re already developing, whatever, a route that you already take to work, we have a preference for sticking with the status quo. And partly that’s because we’re much more tolerant of the possibility of bad outcomes that come from a status quo choice, than we are from switching to something new. And that’s because loss aversion gets recruited for things we’re thinking about starting. And we don’t think about sticking with the status quo as starting. So when we hold onto a stock each day, we don’t think about that as buying anew, it’s just what we’re already doing.

Annie Duke (01:06:02):
This also goes along with something called omission, co-mission bias. We think about switching as making a decision, the co-mission, and sticking as an omission of failure to act. And so what happens is that you’ll hear people say, you’ll say, “Why aren’t you switching?” And they’ll say, “Well what if that turns out badly?” Even in cases where they already know that the thing they’re doing is turning out badly.

Annie Duke (01:06:23):
So I’ve had this happen. I had a conversation with this woman, Dr. Sarah Olson Martinez, who was in a job she really hated. She was an ER doc but also a hospital administrator. And she had really grown to…She was very miserable in the job for a variety of reasons. Many of them had to do with the administrative work coming home with her and interfering with her relationship with her children. And she had another job offer and she was really having trouble deciding whether to switch or not. Should I quit this job I have and go to this new job, which was going to be working for an insurance company. So she described to me how incredibly miserable she was in her work and I was a little confused as to why we were having this conversation, and why she was seeking my advice.

Annie Duke (01:07:01):
And so I said to her, “Well I just want to understand what’s the hang-up here? Why are you thinking that you don’t want to switch?” And she said, “Because what if I go to the new job and I don’t like it and it’s awful.” So I said, Okay, so here’s loss aversion. We can say this, I don’t want to go start this thing because what if it turns out poorly? That’s very classic loss aversion. But it obviously wasn’t being recruited for the status quo, for the things she was already doing. So I just said to her, “Well it’s a year from now. You stay in the job you’re in, what’s the probability you’re happy?” And she said, “Zero percent.” She had enough experience to say that with confidence.

Annie Duke (01:07:34):
So I said, “Okay, if you switch to the new job, it’s a year from now, what’s the probably ability you’re happy?” So, “I don’t know, that’s kind of hard, but probably 50-50.” And I just said, “Okay, so would you rather have a 0% chance of happiness or a 50% chance of happiness?” And then she got it and she switched, but she was so concerned about the 50% chance of being unhappy, that she had been having a terrible time trying to decide whether to switch. So that’s a whole bunch of things. That’s loss aversion, omission, co-omission bias, status quo bias, and also something called ambiguity aversion, that has to do with fear of the unknown, which you could also see in that decision-making.

Annie Duke (01:08:11):
And then we also have the issue, which I think is really important for investors, you have something called an endowment. We value things we own, much more than we value things that we don’t own. And it’s not just ownership over investments, we actually own the stock, or we own the bond, or whatever, we own the option, but it’s also our ideas. And every time we invest, we have ownership over our thesis. And here’s the interesting thing, is that when the thing that we’re doing is out of consensus, this is when it gets really bad. So when we think about these issues of sunk cost, and sure loss aversion, and the way our identity gets wrapped up in things, and the way we have ownership over things, and the way that affects our inability to stop, you have to put a big huge blinking warning sign when the thing we’re doing is out of consensus. So, William, I assume that when you grew up, you thought Pluto was a planet.

William Green (01:09:17):
Yeah, if I cared about it, yeah. I cared more about stock than I did that.

Annie Duke (01:09:18):
If you cared about it. But literally, did everybody else think that too?

William Green (01:09:22):
Yeah.

Annie Duke (01:09:23):
So when there was new data that came in where the scientist said, “Oh, by the way, it’s not a planet, it’s some other thing,” Did you care? Did you change your mind? Did you switch your belief?

William Green (01:09:33):
Yeah, I looked it and I’m like, “Yeah, okay. I didn’t care enough to challenge the belief.

Annie Duke (01:09:37):
Right. And the reason is that it wasn’t part of you, because it was a consensus belief. But when we start to do things that are out of consensus, when we get signals from the world that what we believe to be true isn’t correct, then it becomes very hard to change our mind, because we feel much more ownership over those things that are different than everybody else. Our identity is much more wrapped up into things that are different than everybody else.

Annie Duke (01:10:03):
So there’s this wonderful study that demonstrates this from John Beshears and Katy Milkman that came up from the Wharton School, where they just looked at stock analysts and they looked at 6,000 forecasts or something like that, it was a lot of forecasts. And there were some forecasts that were completely in the mainstream where all the analysts agreed, these were earnings projections. So everybody agreed on the earnings projections for say the next quarter. But then sometimes the analysts would take a really out-of-consensus position about their forecast for the earnings in the next quarter. And what they wanted to know was essentially when you actually see what the earnings report is, and it conflicts with the forecast, what happens? Do they stick to their previous forecast and double down on it for the next quarter, or do they update their forecast? So that’s the question that they were asking.

Annie Duke (01:10:53):
And they found something really interesting that when the analyst made a in consensus forecast and then the actual earnings conflicted with the forecast they made, they changed their minds, they updated their forecast. But when they made an out-of-consensus forecast and then the earnings conflicted with that forecast, they wouldn’t update. They doubled down on it. They escalated their commitment to the out-of-consensus forecast. So there was a really big difference between in consensus and out of consensus, where they stuck to these very extreme positions that they had taken and they wouldn’t change their minds. And I think that we need to realize that which is really an issue about identity, ends up starting to play a very large role in whether or not we’re willing to quit things.

William Green (01:11:37):
I think there’s actually something really profound going on here. And you write about this a bit in the book, this whole issue of identity. And I realize in some ways that I’m a perfect example of it in that buying something like Alibaba or buying Seritage, which is one of the other stocks that I own which crashed during the first day, the early days of COVID because all the mall operators and all the malls shut. And so then I have this friend Monash Powbry who buys 13% of the company and it’s this beautiful contrarian bet. And so with both Seritage and Alibaba, there’s something tribal there. Part of my identity is wrapped up the fact that I like people like Monga and Monash and Lou Simpson. I admire them greatly. They’re contrarian value investors. They’re smarter than the crowd. They have the guts to go against the herd. I see myself smiling as I talk about them. And so it’s part of my identity.

William Green (01:12:34):
And likewise, I think one of the things, I know very little about Bitcoin and cryptocurrencies, and I’m sure people will be writing in to complain about our discussions already, but I think one of the dangers with Bitcoin and cryptocurrency is actually the zealotry where people started to define themselves as being-

Annie Duke (01:12:54):
Very much so.

William Green (01:12:56):
Disbelievers in the system. They saw very rightly, that there was financial and monetary recklessness. And so they were saying, well… This came out around 2008, 2009, during the financial crisis, I understand, where there was this loss of faith in institutions, justifiably. And so there’s something rebellious, subversive, you’re aligning yourself with the tribe of highly intelligent people who understand this technology that most of us don’t understand. And so I think you’re onto something that’s really profound where I think we have to just be very aware of just how vulnerable we are to this self-image.

Annie Duke (01:13:32):
Very much so. On a broad scale, we see people hold very extreme political beliefs. And when the world comes into conflict with those beliefs, they don’t let them go. And you scratch your head, why? But it’s the same reason. You’re part of a tribe. That is your identity.

Annie Duke (01:13:51):
And I’ve done some research with Jay Van Bavel and Diego Rivera and some of our collaborators up at NYU, and what we’ve found is that when people hold a particular belief, if you fact-check them, they will change their belief a little bit, but is completely overwhelmed, and we’re talking in the order of magnitude, overwhelmed by the tribe. We can predict what your belief is by whether you’re like Republican or Democrat. So if we then correct it, both sides, whether you’re Republican or Democrat, you will change your mind a little bit, but it’s completely overwhelmed by the issue of what tribe are you in. And I don’t think that we realize how much that really matters, particularly when we’re talking about these positions that we hold that are quite extreme.

Annie Duke (01:14:39):
And you see this with Doomsday cults. They literally predict the world’s going to end on December 20th. And then December 20th, the world doesn’t end. Are they all fleeing the cult? No. Because again, this is their identity, it’s their tribe. They’ve probably abandoned their families to these beliefs. They’ve given away their worldly goods, and they’ve acted on this in some way that’s very extreme. They’ve staked their identity on this and they won’t let it go. And then you see this behavior, lest you think this is just about politics and political tribe, or even I’m going to be an antiestablishment person who really believes in stable coins or whatever it is, that this is now seeping into the way we make everyday decisions.

Annie Duke (01:15:26):
So one of the things that I think about how this is such a cautionary tale, comes from actually just Sears, the retail store. They were founded in the late 1800s-

William Green (01:15:36):
Which Seritage was spun out of, by the way.

Annie Duke (01:15:39):
There you go. So, full loop. I love this. So Sears is founded in the late 1800s. Start with the book of Bargains, 512 pages, basically riding off the back of the mail system, that infrastructure that was created to get goods to rural America that wasn’t available to them, that were only available in cities. Incredibly successful. I think in the 1920s, Sears, Richard Sears was worth $26 million in the 1920s, a crazy amount of money back then. They eventually get actual store locations, that starts to happen late twenties, early thirties and in order to accommodate the fact that people have cars. By 1950, they represent 1% of US GNP. That’s nutty. And this is a huge company.

Annie Duke (01:16:27):
And then we know what happened. You obviously know what happened. They start to falter, starting in the late 70s as the Walmarts and the Kmarts come along and start pushing them out from below. And then the Nordstroms and higher end department stores start to push them out from above. By the 90s, they aren’t the largest discount retailer anymore. That belongs to Walmart and Kmart at that point and then eventually Target. They merged with Kmart in the 2000s. At that point, Kmart is also faltering in what’s known as a double suicide. That was what one financial observer called it. And then we know that they went bankrupt. Okay, so that seems like a standard, they were around for a long time. Competitors came in, they didn’t do well.

Annie Duke (01:17:06):
But here’s where we really get into this problem with identity. The thing that people don’t generally know about Sears, is that Sears was also a financial services company. So that started because they had a banking division that was offering credit to their customers who were buying in the catalog. But then remember they created these retail locations, as opposed to just the catalog business to accommodate the fact that people had cars. And somebody at Sears had a very good idea, which was, well people are driving these new cars to our stores, we should sell them insurance for the cars. And so they founded a company called Allstate Insurance that I assume everybody has heard about.

Annie Duke (01:17:40):
When I checked for the book it’s market cap was 40 billion. I don’t know what it is today. So this is a very big company that they founded. Originally the debts were in Sears and was only auto insurance. Obviously, it ended up expanding to be more independent and offer liability for anything. And by the fifties, I think it was the largest personal liability insurer that was around.

Annie Duke (01:18:02):
They also acquired a company called Dean Witter, which was a stock brokerage firm, also incredibly valuable. Morgan Stanley bought it. At that time, it represented 40% of Morgan Stanley’s values, very big company. They founded the Discover Card to be able to allow their customers to have credit. That became Dean Witter Discover. And they also bought a company called Coldwell Banker, which merged with another company, Reality. That market cap, last time I saw, was over 2 billion. Again, I don’t know today, but about a year ago it was 2 billion. So these are all very valuable companies that they own. They have a thriving financial services business.

Annie Duke (01:18:37):
And I mean, I assume, William, you’re like, hold on a second. How on earth did these people go broke? Because when you take the market cap of all of those together, we’re talking about billions and billions of dollars, a lot of money. And the answer is, because of identity. Because when they were faced with the faltering retail business, remember the retail business starts to falter in the early 80s, and the shareholders start demanding action, as the retail business is starting to lose money, they decide, and this is in the notes from the board meeting, to get back to its retailing roots. And so they make the decision to sell off all the financial services assets, in order to save the retail stores.

Annie Duke (01:19:18):
Now from the outside, you’re looking in, you’re like, what are you talking about? You have bad assets here, these stores that are being pushed out by competitors and aren’t making any money. You have this thriving financial services business. Why aren’t you getting rid of the stores? And the reason is because Sears was a retail company. They sold stuff, and they wanted to protect that identity. And we know what ended up happening. You of all people know what ended up happening with that.

William Green (01:19:44):
Yeah. And you are adding in things like endowment effect, and sunk cost, identity. Again, it’s a lollapalooza of these cognitive biases.

Annie Duke (01:19:52):
That’s right.

William Green (01:19:53):
And I think what’s interesting, Annie, also is, you mentioned in the book that knowing is not the same as doing. And that knowing, for example, about sunk cost effect doesn’t protect you from it. And I was looking over many of the questions that came in when I asked people on Twitter to send in questions for me to ask you. And someone called @Fixias14, who’s also called Michael Field says, “Well, how does one actually internalize the concepts discussed in the book, as opposed to just being able to articulate them?”

William Green (01:20:21):
And I think that’s a hugely important question. We can listen to all of these discussions of cognitive biases. I mean, I can actually look at Alibaba and say, Okay, I’m down 59%, here are all of the biases that are making me stupid, and I still can’t decide what to do. So, how do you shift from knowing that you are irrational, and that all of these things are happening to you, that you are subject all of these idiotic cognitive biases, to actually internalizing something and doing anything about it to protect yourself?

Annie Duke (01:20:57):
That’s a good question. So the first thing is, we talked about, one of the things is kill criteria. So, in think in advance about what-

Annie Duke (01:21:03):
… the things is kill criteria. Think in advance about what are the signals that I might see in the future that would tell me that it’s time to walk away and you will get better at it. Now even in the case of Alibaba, so you’re already in it, you didn’t do this in advance, but it’s not too late because what you can do, and hopefully you will when we get off, is sit down and say, “What do I need to see within the next quarter or the next two quarters from the way that this investment might perform?”

Annie Duke (01:21:24):
Essentially think, “How long can I tolerate this, or how much time do I need in order for me to actually get the information that I would need in order to be able to make a decision?” Figure out what that time period is and then figure, at the end of that time period, “What would I have to see? What are the benchmarks that this thing would have to hit in order for me to feel like I ought to continue? And if it doesn’t hit these things, then I should walk away.” Now you can certainly get help on trying to figure out what those benchmarks are from somebody who’s very experienced, who’s going to be a good advisor to you.

Annie Duke (01:21:51):
And that brings us to the second thing, is get yourself a quitting coach because you’re in it. You’ve lost money on the investment. You made the original decision to invest in it, but go find somebody who didn’t make that decision along with you, who can really help you to see things from the outside. Because I’m sure you’ve seen people in these situations, where they’re holding onto investments where you’re going, “How can they not see that’s a dog? They should have sold that a long time ago.” Well, trust me, people are looking in on you and thinking the same thing. So go find them and tell you to help you.

Annie Duke (01:22:20):
And something that’s important about this is to understand, I’m sure you have lots of very smart people in your life and you may have the intuition, “Well, would they tell me if they see it?” And the answer is, no, they wouldn’t, and in the same way, you don’t say it to somebody else because you don’t want to get yelled at or make them feel bad or ruin your friendship. So it’s-

William Green (01:22:36):
And you might be wrong as well. I mean, I had this discussion with Thomas Russo, who’s a famous international investor, a couple of weeks ago. And he’s like, “No, I was wrong about Alibaba and I started to sell my position, I’ve trimmed it. I didn’t understand just how much risk there was of the Chinese government messing with this stuff.” And I said to him, “I sort of have a five-year rule where I just hold stuff for five years so that I don’t make so many decisions.” And he is like, “Well, you will probably do better than me.”

William Green (01:23:04):
And that’s what’s also so hard is, it’s so murky. It goes back to your question, your conclusion from Poker, which is, “I’m not sure. There’s so much uncertainty.”

Annie Duke (01:23:15):
Yeah. I would say, first of all with the quitting coach, you want to sit down and create these kill criteria together so that’s something that you can do. Is this a situation where I should just hold for five years? Because I think we should agree, you shouldn’t hold everything for five years. Let me give you a simple example. For some reason you decide to buy a meme stock, are you supposed to hold that for five years? Well, obviously not. If something goes to a penny, if the business is clearly going out of business, hopefully you notice that beforehand. So we can have a rule of five years, but we have to have some unlesses that are attached to that.

Annie Duke (01:23:50):
And this is true of all goals that we have. So we want to set out kill criteria, ideally with a quitting coach that we can go and check in, someone who’s very experienced, who can help us look at those criteria, revamp them. Obviously if you make kill criteria for a particular quarter, then you would want to make a new set of them. And then get somebody to help you to be able to see it from the outside. And then when it comes to goals, hard and fast rules, I only hold for five years. Make sure that you have some unlesses that are attached to that.

Annie Duke (01:24:21):
And let me explain why. This isn’t going to seem related, but it is. There’s this woman, Siobhan O’Keeffe, who was running the 2019 London Marathon. And right around mile four she started to experience a lot of pain in her leg. And then on mile eight her fibula snapped. She literally broke a bone in her leg. Now I assume you share the intuition that she stopped running one would assume, but she didn’t. She finished the race.

Annie Duke (01:24:49):
Now this is weird. She broke her leg, she was in excruciating pain. All of the doctors were telling her, “Don’t run.” The people in the medical … Like, “Stop running. You shouldn’t be running.” And you can see here where she’s risking never being able to run another race again. What if it becomes a compound fracture? It’s going to obviously take a lot longer to heal a broken leg that you’ve now run an extra, what was it, 18.2 miles on. But she kept running nonetheless.

Annie Duke (01:25:15):
And this is part of the problem with sticking to things too long, is that it stops us from having the opportunity to do other things. We can’t use that time and attention for other stuff. Now less you think that she’s strange, there were three other people in the same race who basically did the same thing. And if you look at any marathon, there are people just running with broken legs all the time.

Annie Duke (01:25:34):
So the question becomes why? Why are they doing that? Separate and apart from the sunk cost issue and whatnot. And it’s because there’s a finish line. The finish line is 26.2 miles. And when we talk about being in the losses as a cognitive phenomenon, it doesn’t matter that she ran eight miles, she’s short 18.2 miles, so therefore she has to keep going because otherwise she has failed. If you get 300 feet from the top of Everest, you didn’t climb 29,000 feet, you fail. You failed to get to the finish line. So when we set these goals, these marks, these lines in the sand, the problem is that we will keep going toward them basically no matter what, because otherwise we will have failed.

Annie Duke (01:26:18):
This would be the caution with, “I have a five-year hold,” which is unless. An example might be, “Here’s my thesis. I have some sense of the effect, for example, of what the Chinese government is going to have on this particular investment. So as long as that holds, as long as the status quo is the same and nothing tells me that that was a silly assumption, I’m going to hold for five years. I don’t care if it goes up or down or whatever. I’m just going to hold for five years.”

Annie Duke (01:26:49):
Unless I see that Jack Ma disappears. At which point I should say, “Maybe I don’t know so much about the Chinese government, and maybe they’re going to really interfere with investments here. And I should get out of this because there’s something that I thought was true of the world that I now have new information that isn’t true of the world. And while normally my five-year hold is a rule that I really try to follow in order to protect me from overreaction to the ups and downs of the way things are going and I’m trying to ride the base rate, I want to make sure that I’m not missing out on that. Which is that things tend generally go up and I want to have enough time to do that.” Sometimes there’s going to be something that happens where I should put this big “unless” to that goal.

Annie Duke (01:27:36):
So what I would say with something like Alibaba right now is to just sit down and say, “What are the things that I would have to see over the next quarter that would tell me that I have to change my mind?” Or maybe go talk to somebody and say, “How bad do you think it is that there’s been this interference from the Chinese government?” Someone you really trust and find out what they think. And then no matter what, you’re going to be making the decision under uncertainty.

Annie Duke (01:27:59):
The question that I would have for you is not so much, are you going to do better than that investor, but if you had talked to him today and you knew what you knew about Alibaba today, you knew what happened to Jack Ma today, would you buy it today on his advice? And if the answer is no, then you shouldn’t be holding it, even if you have bought it in the past. And I think that’s where we have to start to change the way that we’re approaching these problems.

William Green (01:28:24):
I think also, Annie, you bring up this really important point that’s a thread through all of your work.

Annie Duke (01:28:28):
And wait, I just want to say, that’s not stock advice for anyone. I’m not advising anybody to sell Alibaba, nor buy it. I’m not.

William Green (01:28:37):
Yeah. And likewise with our cryptocurrency expertise or lack thereof-

Annie Duke (01:28:42):
Yes, exactly.

William Green (01:28:42):
… this is just an example of the kind of-

Annie Duke (01:28:43):
We’re just chatting.

William Green (01:28:44):
Yeah, we’re just talking really about the difficulty of thinking rationally given our cognitive biases. And one of the things you talk about that’s a recurring theme in your work is the importance of bringing in other voices, bringing in other perspectives of … And this goes back all the way to when you were learning poker, where you would be listening to people like Erik Seidel or to your brother, Howard.

William Green (01:29:05):
Likewise, I see it in investing, where for example Howard Marks has Bruce Karsh as his partner. Buffet calls Munger the “abominable no-man,” because he says no to so many of the things that Buffet thought were really good ideas. I see people like Mohnish Parbrai and Guy Spier talking about their ideas. And it doesn’t always hold. I mean, Guy I think thinks Mohnish is taking way too much risk putting lots of money in Turkey, and Munger often makes these great points that then Buffet ignores. It’s not like have to take the advice.

William Green (01:29:37):
But this idea of actually having somebody who you trust who … As you say, Conoman, I think Danny Conoman is one of the smartest guys on earth who you’re friends with. Has Richard Thaler-

Annie Duke (01:29:48):
Yes, they all were.

William Green (01:29:48):
… another Nobel Prize-winning economist. Can you talk about-

Annie Duke (01:29:52):
I’ll take them.

William Green (01:29:53):
Yeah, can you talk about the fact that Conaman actually gave explicit permission to Thaler to tell him what he doesn’t want to hear? Because that also seems to me really important.

Annie Duke (01:30:03):
Yeah. I think that we think that people are going to tell us what they see, particularly if they love us. And we assume that people who love us have our long-term best interests at heart. So if there’s something that we should be stopping … Conaman says, “If I’m pursuing some sort of line of research, whatever, I assume that people will tell me that I shouldn’t be.” But nobody wants to do that. Nobody wants to say, “Oh by the way you’re failing,” or, “The thing that you chose to do isn’t working.” Or, “By the way, you should break up with this person.” God forbid that they actually don’t and then they know you think that they shouldn’t be with that person anymore.

Annie Duke (01:30:36):
But separate and apart from just that kind of danger, there’s also, we have a tendency with people we love to want to cheerlead. Look, if you’re talking to me and you say, “I have a five-year rule and so this is why I’m sticking to this,” my tendency, less anything else, is going to be like, “That’s great. You’re so disciplined and want to cheerlead you to …” Because I want you to feel good. And I don’t want to be the abominable no- man.

William Green (01:31:01):
And just to be clear, before I sound even more stupid than I am with that decision, really what I was trying to say is, once I buy something, I’ve got to hold it for a minimum of five years. Because I’m like, one of the biggest problems investors have is that they’re too impatient. And if I just tie my hands, then even though I’ll be wrong sometimes, over time I’m pushing myself towards good behavior of deferring gratification.

Annie Duke (01:31:23):
Listen, I completely agree. I don’t want people to start making individual decisions. All I want is that when you say, “I’m buying it, I’m going to hold it for five years,” that you just add some key unlesses. That’s it.

William Green (01:31:36):
Yeah, it’s a great point. Yeah.

Annie Duke (01:31:37):
“This is my general strategy, but let me remember why I bought this. So if there’s something that really changes, then-”

William Green (01:31:48):
You know, Annie, there’s a really beautiful essay that George Orwell wrote on writing, on how to write. And he lists all of these rules and the final rule he says is, “Ignore all of these rules rather than do anything downright barbaric.”

Annie Duke (01:32:02):
There you go, so that’s … Yes.

William Green (01:32:04):
But it’s such a beautiful “unless,” isn’t it?

Annie Duke (01:32:05):
That’s the perfect “unless.” In fact, Churchill, when he said, “Never, never, never, never, never, ever give up,” he said, “Except in matters of honor.” So he even had an “unless” on that one when he was telling people never to quit. He was like, “Except when you’re supposed to,” basically is what he said.

Annie Duke (01:32:20):
Yeah. I think that the thing is that what we want to do is make a distinction between is this a situation where that “unless” should apply, for example with you, or is this a situation where I’m just panicking and I’ve got this rule and the rule is in place for a reason. I have a similar role by the way, as you do, which is I want to make sure that I’m realizing those gains and not overreacting or thinking that I know more than I do, for example.

Annie Duke (01:32:45):
And what we need to do is find someone that we can trust to tell us the truth. When they see that things aren’t going well for us, when our job isn’t going well, or a relationship isn’t going well, or research isn’t going well, or we’re so sure of this NFT that we own, or that NFTs are the next thing and we’re literally taking all of our money and putting it into NFTs. Don’t we want someone to stop us? I mean, just strictly from a risk standpoint. Don’t you want someone to say, “Don’t do that. Don’t take all of your money out of here and put it all over into one asset class. That’s nuts.”

Annie Duke (01:33:19):
But people tend not to step in and do that, unless we give them permission. And that’s the key thing, is that we have to tell somebody, “I don’t want you to cheerlead. I don’t want you to support. I don’t want you just to bite your tongue when you see that I shouldn’t be doing this. I actually want you to tell me.” And you have to be very explicit in giving them permission to do that.

Annie Duke (01:33:39):
Because what you have to say to them is, “I recognize that I’m going to make a lot of decisions that are bad for me in the long run, that I’m going to rationalize away or they’re going to feel good in the short run or whatever it is. And I need you to be on the right ride with me, and I need you to tell me what’s good for me in the long run, even if it hurts in the moment. So please tell me the hard things. I promise I will not be mad at you. I promise that this will be really important feedback for me. I promise that I will try to listen, I won’t always follow what you say. But I know it’s going to make me better for it.” And if you can do that with someone and find people in your life who are willing to enter into that game with you, you’re going to do a lot better.

William Green (01:34:18):
One thing that’s really striking when I look at your life, Annie, is that you’ve been helped by an extraordinary array of unbelievably talented people. I mean, obviously, it goes back to-

Annie Duke (01:34:30):
Lila.

William Green (01:34:31):
… Lila, who we talked about at the start, but also when I look at the acknowledgment sections of your books I can see, talk about Michael Mauboussin introducing you to Sasha Cohen-

Annie Duke (01:34:43):
Ah, love him.

William Green (01:34:44):
… who’s someone you write about in this book, or talking through the book with you in the early days of the book. You talk about getting help from people like Danny Conoman and Richard Thaler in developing the ideas in this book. So two incredible Nobel Prize-winning economists. You talk about Philip Tetlock, the author of Superforecasting. You talk about getting Charles Duhigg, who wrote this great book, The Power of Habit, lending you his book proposal so you could see how to write a book proposal. Dan Ariely, the great expert on pain and the like, introduced you to your agent Jim Levine, who’s also my literary agent, who’s amazing.

William Green (01:35:20):
And I think also of Erik Seidel helping you. Or why is it that so many extraordinary people have been so generous with you. I’m sort of asking this, it is difficult for you to be honest about it and self-effacing about it.

Annie Duke (01:35:37):
I think because I ask.

William Green (01:35:38):
Really?

Annie Duke (01:35:40):
I think. I don’t know. I don’t know. Michael Mauboussin in particular has been so incredible and generous and I just really enjoy talking to him so much. And I guess maybe he enjoys talking to me, I don’t know. I sort of pinch myself all the time. I mean, Phil Tetlock is not just a collaborative of mine, but he’s now my PhD advisor and I’ve known him for 10 years. I was one of the people he talked to for Superforecasting, so we developed a relationship back then. And his wife Barb is so incredible and incredible researcher in her own right.

Annie Duke (01:36:16):
Yeah, I mean, first of all, I do ask/ I think maybe I’ve been lucky enough. I mean, some of it I think is luck. Obviously, I mean, there’s a lot of hard work to get into that program and have Lila be my advisor, so that I would less call luck. But I think that I was really lucky to end up in poker, and I think people think it’s kind of a cool thing that I did. And I think that that gets people to talk to me, and then it’s my job then to make it interesting. And at least when it comes to this topic of quitting, what I found as I was reaching out to people … Don Moore I would include in this as someone who’s been incredibly supportive and amazing. He wrote Perfectly Confident. David Epstein, who wrote Range.

Annie Duke (01:36:59):
Is that they were excited about the topic. So in some ways I think that’s also just a nice thing, is that the kind of thing that I was really interested in thinking about, which was like how do we solve these problems of quitting when that decision is made under uncertainty itself, just that people were on that ride with me in thinking that this was a fun, cool, important thing for somebody to write about. And so they were very generous with me about that.

Annie Duke (01:37:24):
But I had to ask. And I think this is really important, just ask. What’s the worst that happens? They say no, whatever.

William Green (01:37:31):
Maybe it goes back to loss aversion. We have this fear that the pain of loss is going to be twice as intense as the pleasure of gain. And so we’re afraid to ask.

Annie Duke (01:37:40):
Right, exactly. That fear of getting said no, but what comes out of it is so amazing. And then once people are excited about the topic, they then start thinking about it. And Don Moore, I had a conversation with him about Quit, and he ended up sending me a three-page thing with a whole lit review of things that he thought were relevant. And I mean, how generous, how incredibly generous with your time and attention that you would send me that. And then he read an early version of the book and said, “I really think you need a section that’s just on over-optimism.” And he helped me with that and worked on that with me, and it definitely made the book a lot better.

Annie Duke (01:38:17):
I don’t know. I don’t know. I really don’t know why I’m so darn lucky to have these people to talk to. But thank God I am because I’m not smart enough to write a book without them.

William Green (01:38:29):
You wrote in Thinking in Bets about the superlative players who, quote, bestowed their expertise and friendship, a truly wonderful gift. And I just was sort thinking about that, it’s so central to the way that you learn obviously is by getting a peer group of people who were really intellectually engaged. I guess that’s one of the common denominators, is both with poker and with your research and your books, there’s an intensity to your curiosity about the subject that people probably feel. And so they engage with it and it’s cool, interesting work. So these are all people who are actually intellectually very engaged.

William Green (01:39:08):
And so I think that’s part of it. But I also, if I’m honest about it, I think part of it’s just if you’re a decent and amiable and a nice person. There’s a weird, competitive advantage to being nice. Does that-

Annie Duke (01:39:24):
Maybe, yeah. I guess, I don’t know. The thing is, I really, I learned a long time ago in poker that confidence does not mean thinking you know everything. And in fact, if you do, then you’re not going to do well. You can be pretty confident that in that moment you’re playing well and going to … I was pretty confident I could write a reasonable book about quitting, but in no way did I feel like I knew anything. If I have the chance to talk about Richard Thaler, who was the first person to identify the sunk cost effect as a general phenomenon, I’m going to spend all the time that he’s willing to give me and hopefully soak up everything that he has to say, because I know it’s going to make what I’m doing much better.

Annie Duke (01:40:08):
And I’ve come across people in my life who you can sort see that they’re borrowing ideas from other people but then not acknowledging it. They’re acting as if those are their own thoughts. I don’t know why, I think maybe because they think they want to appear to be smart. And I’m someone who’s just like, “That wasn’t my idea. That was Mauboussin’s idea. Thaler told me that. Conoman told me that.” Because what’s wrong with that? That’s the thing, is that I really truly believe that, I’m like …

Annie Duke (01:40:40):
With my books I say, I don’t think I have an original idea in any of the books that I’ve written. I think I put the things together pretty well. I think that’s the thing that I do pretty well, is to put ideas that have obviously come from people who are much smarter, think much more deeply about these topics than I possibly ever could, have done the original research, really identifying these phenomena. And then I think I put it together in an interesting way.

Annie Duke (01:41:04):
And I’m totally okay with that. That’s fine by me because I’m excited that I get to, in some way, translate the work of these people who are so incredible in a way that’s nicely consumable for the average person. I think that’s an important role to play and I’m perfectly happy playing that role. It’s a role that I really enjoy and I get to learn in the process. I get to explore the ideas in the process, and I become smarter in the process. And isn’t that lucky for me?

William Green (01:41:33):
It is lucky for us as well. It’s a very cool book. And I have to say, before we go, one of my favorite parts of it is tucked away in a footnote where there’s a perfect example of this, where you use synthesize this stuff from Niels Bohr and Thomas Mann from an essay that he wrote in 1928 about Freud. And they’re not necessarily new ideas, but it’s also something that Philip Tetlock said about how the opposite of a virtue is also a virtue. And this-

Annie Duke (01:42:01):
Yeah. And this is, people, you have to read the footnotes to know that that came from him, so I appreciate that.

William Green (01:42:06):
But there’s something beautiful about that synthesis, and you synthesized this idea and it expressed something that I’ve been thinking about a lot but that you’ve drawn together from these different sources. And it’s a very valuable thing. And I’ll just say, I was telling my daughter on Facebook yesterday about this, she’s in college in Boston. And she took a picture of me looking incredibly excited, holding a spatula as I was cooking my oatmeal, because I was telling her-

Annie Duke (01:42:30):
Oh my gosh, I’m so excited.

William Green (01:42:30):
… “This is what Niels Bohr said.”

Annie Duke (01:42:31):
Well, thank you so much.

William Green (01:42:32):
So no, I think you’re doing something very valuable in taking your own experience from poker and investing and cognitive bootstrapping and the like, and then drawing in lots of insights from other people and from the reporting you’re doing. And it’s great. It’s a really interesting and thought-provoking book and so was thinking about, and so I wish you much success with them. I’m sure this will do great.

Annie Duke (01:42:55):
I really appreciate it. Thank you so much. This was such a great conversation.

William Green (01:42:59):
Ah, thanks. It’s been a great pleasure, and I hope we’ll get to meet in person before too long. It’s been really fun.

Annie Duke (01:43:04):
Awesome. And by the way, I always say this, edit away. I don’t care. I know I’ll sound better if you edit me.

William Green (01:43:10):
You sound great. Annie-

Annie Duke (01:43:11):
Nothing is sacred.

William Green (01:43:12):
… best of luck with the book and everything else. It’s been a real pleasure chatting with you.

Annie Duke (01:43:16):
Okay, awesome. Thank you so much, William.

William Green (01:43:17):
Take care.

Annie Duke (01:43:17):
This was really fun.

William Green (01:43:18):
Thanks.

William Green (01:43:19):
All right folks, thanks so much for listening to this conversation with Annie Duke. Had an absolute blast chatting with her and I hope you enjoyed it as much as I did. If you’d like to learn more from Annie, I’d definitely recommend her bestselling book, Thinking in Bets. Also, really enjoyed her new book, which is titled, Quit: The Power of Knowing When to Walk Away. Meanwhile, I want to thank everyone who suggested questions over Twitter for me to ask Annie.

William Green (01:43:43):
For each episode of the podcast, I like to send out one signed copy of my book, Richer, Wiser, Happier, as a way of saying thanks for all of your excellent questions. This time around the prize winner is a listener in New Jersey who goes by the Twitter handle @phyxius14. Please feel free to follow me on Twitter @williamgreen72, and do let me know how you’re enjoying the podcast. I’m always delighted to hear from you. I’ll be back very soon with some fascinating guests, including the Nobel Prize-winning economist Robert Shiller, who wrote a famous book on Irrational Exuberance, and a renowned investor named Francois Rochon, who’s beaten the market by an enormous margin over the last 30 years. In the meantime, take care and stay well. Thanks a lot for listening.

Intro (01:44:27):
Thank you for listening to TIP. Make sure to subscribe to We Study Billionaires by The Investor’s Podcast Network. Every Wednesday we teach you about Bitcoin, and every Saturday we study billionaires and the financial markets. To access our show notes, transcripts, or courses, go to theinvestorspodcast.com. This show is for entertainment purposes only. Before making any decision, consult a professional. This show is copyrighted by the Investor’s Podcast Network. Written permission must be granted before syndication or rebroadcasting.