DeLay your intuition. Don't try to form an intuition quickly, which is what we normally do, focus on the separate points and then when you have the whole profile, then you can have an intuition and it's going to be better.

Hello and welcome. I'm Shane Parrish and this is the Knowledge Project podcast, exploring the ideas, methods and mental models that help you master the best of what other people have already figured out. To learn more and stay up to date on new episodes, go to First Stop Blogs podcast. I also put together a weekly newsletter that you'll love. It's called Brain Food, and it comes out every Sunday, much like this podcast. It's high signal, low noise.

It's timeless. And Mind-Expanding read what you're missing at F-stop Blogs newsletter. Today, I'm speaking with Daniel Kahneman, emeritus professor of psychology at Princeton, who received the Nobel Prize in economics in 2002 for the work he did on decision making with Most Diverse. Danny is the most influential living psychologist, a true legend in his field, and this conversation was a great honor publicly. He's probably best known for his book, Thinking Fast and Slow, and his work on drawing attention to our cognitive biases.

Our conversation revolves around how to make better decisions, our intuitions, what, if anything, we can do to reduce our cognitive biases and how rules make great defaults. It's time to listen and learn. The Knowledge Project is sponsored by Medlab for a decade, Medlab has helped some of the world's top companies and entrepreneurs build products that millions of people use every day. You probably didn't realize that at the time, but odds are you've used an app that they've helped design or build apps like Slack, Coinbase, Facebook Messenger, Oculus, Lonely Planet and many more.

Medlab wants to bring their unique design philosophy to your project. Let them take your brainstorm and turn it into the next billion dollar app from ideas sketched on the back of a napkin to a final ship product. Check them out at Medlab Dutko. That's Metal Abaco. And when you get in touch, tell them Shane sent you. Down, so happy to get a chance to talk to you. Well, I'm happy to have you here. What was your childhood like?

What were you like as a child? Oh, my God. That was a long time ago. I was I was an only child, as you might expect. I suppose I was I thought I'd be a professor when I was like three or four years old because people told me it would be because I probably spoke with long words and stuff like that. So and then the rest of my childhood, I mean, I was five when World War Two began.

So and that was a Jew in France. So I've had a difficult childhood. But from that point on. But but I was so lucky. I was I was a nerdy trial. I was quite inept physically. Very fortunately for me, when I finally moved to Israel at age 12, they held me up a grade and them and that was all right. But that's that's what I was like.

If there are any particular lessons or memories that stand out for you, there are two of them that they speak about.

So one is that I was I was a psychologist very early on. It was very clear. I wrote an essay and before I was 11, I remember where because it was it was a German counterattack. It was during that period we were in Paris and I wrote an essay about faith and religion. And it was a very pompous and see, I had a little book that was titled What I What I write about, what I think something like that.

But the essay started with another pompous thing that I quoted Pascal. My sister had passed through exams and I had read novitiates, studied some Pascal and I had read it. And Pascal had said that faith is God made sensible to the heart. And, you know, a little me. I said, how true. That's that my they said, and then but then I said, But faith is really hard to get. You don't sense God all the time.

So that's what religious pump is for cathedrals. Organ music they give you and I call that OSAT face sort of substitute faith because it's a it's a similar feeling. It's got to do with God. And that's what you must do with that. That's a psychologist. So it's clear that, you know, that was my calling. And so that's one significant memory of my childhood.

You born to be a psychologist? I think so.

I think so. I mean, I you know, it's always had that point of view that later as a teenager, it was, you know, interested in all the philosophical issues like, you know, does God exist and what's good and bad and stuff like that, then why shouldn't we must obey, you know, serious questions. But but I discovered that actually I was less interested in the question of whether or not God exists than in why do people believe that he exists.

That I thought was interested and I wasn't particularly interested in the question of what's good or bad, but that was really interested in what makes people angry and indignant. So, you know, I've had the psychological point of view since then that since my childhood. Was there anybody who sort of influenced you to go on to study this? I mean, it's one thing to have these dreams is like a twelve, thirteen, fourteen year old boy. It's another to turn this into, you know, probably the most eminent career that's ever happened for a psychologist.

No, not the most dominant career, you know. And I wasn't sure actually that I would do psychology when I took a vocational exam to tell me what I was good at. And psychology and economics turned out. But, you know, that was unexpected. I was and then I took psychology as an undergraduate and mathematics at which I was not particularly good. So and no, it's not that I knew at the time that, you know, that calling to be a psychologist then occurred to me.

I thought, you know, I thought I'd be a professor in one thing or another. I mean, I thought I'd be an academic, but not psychology specifically.

You worked for with Amos Tversky for a long time. Are there any particular stories that you remember about working with him that bring a smile to your face?

Almost. Everything about working with them brings a smile to my face. You know, he was a very unusual person. Most people who knew them thought that he was the smartest person that ever met. And in fact, a famous psychologist, Nick Nesbitt, said that it's sort of an intelligence test. When you said that when you're with them, how long does it take you to figure out that he is smarter than you are and the first to figure that out the smarter you are.

So he was he was super bright and very, very funny. You joked a lot. He laughed a lot at his own jokes. And that was infectious. When I was with him, I was very funny to more than half the laughter. I love my lifetime. I've had during the 10 years I worked with him.

You have an interesting distinction between happiness and satisfaction. Can you walk us through that?

Yeah, sure. I mean, the word happiness is so ambiguous and it means so many things to many people. But one sensible interpretation of it is that it's got to do with your emotions, with how you feel, with the emotional tone of your life, whether it's a happy life, you know, it's pleasant to be you know, satisfaction has a completely different thing. I mean, life satisfaction is how you feel about your life when you think about your life.

And most of the time you don't think about your life. You just live. But, you know, sometimes the sort of bloke and that's that's when you determine how satisfied you are. That's life satisfaction, not satisfaction.

It's life satisfaction to rebalance the two. Or how would you think about them? Should we be happy when we're younger, more satisfied when we're older?

That thought had never occurred to me when I began to work on that. So I started out thinking that happiness and that sense of how you feel when you live, that that was reality and that life satisfaction was just stories that people tell themselves. And the important thing was to be happy in real time. But later, when we did more research, it turned out that the circumstances that make people happy and the circumstances that make them satisfied with their life are not the same.

So happiness is mostly social. It's, you know, it's being with people you love who love you back. That's that's a lot of what happiness is. Life satisfaction is much more conventional is to be successful. And, you know, so it's money, education, prestige, that sort of thing is what this election is about. So those are two very different things. I thought that was satisfaction is irrelevant. You know, that's how I began.

And we we have a research program where we were we were trying to, you know, to show that this is the case. But then after a few years, I realised that what people really want in their life is they don't seem to care about how happy they'll be. They seem to want to be satisfied with my life. They seem to want to have a good story about their life. And then I was in the position of saying that to define wellbeing in a way that people didn't seem to care particularly about.

So that was not a tenable position. So I I dropped back into saying that I had no idea how to deal with it.

Was this a result of the research you did? Some research that was I think it said above seventy thousand. You don't become happier, but do you become more satisfied?

No. The research I did with Angus Deaton at Princeton, famous economist, we showed that in terms of happiness, in terms of emotional toll positive and negative, having a lot of money doesn't make you happier, but being poor makes you miserable. So that's above the threshold that was like seventy thousand dollars approximately in the US. Then extra money didn't make you emotionally happier. But with life through the factory, it was a different story with lots of destruction that doesn't satiate sort of it's always good to have more because basically I think money is a proxy for success and it's a proxy for subjective success in many cases.

So it's not necessarily about spending it or doing something, but no, it's just about just getting it.

I mean, you know, you look at all those people, all those billionaires working their heads off and they're clearly not doing this because they need more money. They're trying to get more money. They're trying to get more money, because that would be an indication that they're good at what they do. I think mostly it's a proxy either of those variables correlate to longer live in happiness or satisfaction. Both, apparently. But, you know, it's it's hard to separate.

I haven't been followed, you know, shortly after the flooding. I don't know what well-being was. I sort of stopped doing research on this. So I haven't been following. But I think there's clear evidence that being effectively happy is really good for you. And you do live longer and better.

And so a life satisfaction works in the same direction, whether it's separable, which of them, you know, is it more important that I don't want to switch gears a little bit and talk about behavior and I'd love your insight or expansion upon the idea of we can change behavior and how do we go about changing our behavior?

Well, you know, I'm not sure I buy the premise. I think changing behavior is extremely difficult. The there are a few tips and, you know, few guidelines about how to do that. But anybody who is very optimistic about changing behavior is just it's it's hard to change. Other people's behavior is very hard to change your own.

No, simple. This is what marriage is all about. Right. Among other things, other people when when, you know, married people are try to change each other's behavior. A lot of satisfaction not on their way to a good marriage.

I think we'd all be happier with lower expectations. I mean, and even if you have expectations, don't try to change because, you know, it's very unlikely to work in a significant way.

I can think of the common ways that we would sort of go about behavior change and it would be, you know, making good behaviors more easy or negative behaviors harder.

Then that's the main, the main. You know, when you want to influence somebody's behavior. That's a very big influence. I've always thought that this is the best psychological idea ever. You know, so far as I'm concerned. But it's that when you want somebody to move from A to B in terms of their behavior, you can think of it that there are two ways of doing it. You can push them or you can ask the question, why aren't they doing B already?

Which is an unusual question that you want. So then when you ask why, why not? Why aren't they doing B they ought to have. I think they ought to. Then you get a list of words, OK, you win that psychologists and my guru and that's my hero. And many people say here he spoke of restraining forces. I mean, so there are reasons why they're not where you want them to be. So he spoke of behavior as an equilibrium, the forces of pushing you one way forces that are pushing you the other way.

So how loud you speak, how fast you drive? It's easy to think of it as an equilibrium. And what we tend to do when we want to move people from A to B is we push them. We add to the driving forces and called Lewin's insight was that this is not what you should do, you should actually work on the restraining forces and try to make them weaker. And that's a beautiful point. And he showed he had that image that, you know, I've had since I was an undergraduate.

And I'm not sure actually whether it was his image or something that I drew from reading him. But it's like you have the plank and it's being held by two sets of screens. You know, you wanted to move one direction and so you could add another string that would push it that way. I could remove one of the springs that are holding it back. And the interesting thing, and that's the striking outcome is when he moves, if it moves because of the driving force you've added to the driving force, then at equilibrium it will be in a higher state of tension than it was originally.

That is because you've compressed wet spring and such, pushing back harder. But if you remove a restraining force at equilibrium, there would be less tension of the system. Must have been 20 years old. I thought that's just so beautiful.

What do you wish that everybody knew about psychology? That you don't think that they do? If that was class one, what's class to, you know, class two, which is the development from class one? You know, it's it's the same idea. Extend the past to that. Behaviors don't necessarily reflect the personality, but behaviors have a lot to do with the situation. And so if people behave in strange ways, look at the situation there and what are the pressures in the situation that make them this way.

So there is a bias that the social psychologists world on social psychologists call the fundamental attribution error. And that means that when you see people acting in some way, you think that it's because of their personality that they do. It may not be the case. It's quite likely that the situation is making them do. I'd like people to know that motivation is complex and that people do good things for a mixture of good and bad reasons, and they do bad things through a mixture of good and bad reasons.

And I think that if there is a point to educating people in psychology is to make them less judgmental, just have more empathy and more patience. And being judgmental doesn't get you anywhere.

When you talk about a situation on one of the things that comes to mind as it's so easy for us to give our friends advice, but if we were in that situation, we might not necessarily see it. Why is that the case? Why is it so much easier to give other people advice?

I mean, feelings get in the way of clear thinking. There is a phenomenon that we call the endowment effect, which is that when I ask for more money to sell you my sandwich than I'd pay to get it. I mean, that's essentially the endowment and our explanation of it. There are many explanations, but the story I like to tell about it is that it's more painful to give something up than to get something. But there is an interesting result that if you have an agent making decisions on somebody's behalf, that agent doesn't have lost a vote so that they themselves and buys at the same price, which is the economically rational thing to do where this goes into policy and governments and really important things that governments are like agents or people who think about the good of society and agents.

They take the economic view. They take the view of what things will be like at the end. They don't figure out that there are some people are going to be losing because of the reform that they make. And it turns out that you can really expect losers, potential losers, to fight a lot harder than potential winners. And that's the reason that reform so frequently fail and that when they succeed, they're almost always way more expensive than anticipated. They are more expensive because they have to compensate the losers.

And that frequently is not anticipated. So that's an example of a story about that incorporates behavior change and and the difference between perspectives, between being, you know, in the situation, feeling the pain of giving up a sandwich and not feeling the pain of giving up the sandwich. That would have huge public policy sort of implications to. Right. That we don't tend to think about or discuss. That's really interesting. I know there I'm going to come back to sort of Situational Decision-Making based on sort of like what we see is all there is.

And we have these feelings that we can't sort of. Disassociate. Well, how does the environment play a role like the physical environment in sort of what we decide or does it?

I mean, you know, there are so sort of obvious things that we know people are hot and bothered distract them and there is a lot of noise and so on. And they'll think this well, that we know that's not even there. There are puzzles. I mean, many people think and work a lot better in cafes, you know, where there is actually ambient noise and activity around them and it helps them concentrate. But also there's a very simple story of the environment.

But certainly you can make the environment tough enough so that people won't be able to think properly. That's that's feasible are the things that we could do to, I guess, push the environment to be more conducive to clear thinking, the physical environment in this case?

Oh, those sorts of odd findings. You know, the colour of the colour of the room, the sun colours are better than others. You would expect that. So the colours are more calming than others. So you wouldn't want to be in a red room making decisions like making decisions. But, you know, those are extreme and minor effects.

I want to come to intuition and noise later. Is there anything else that stands out that gets in the way of clear thinking that we can sort of bring to the surface now?

Well, you know, what gets in the way of your thinking that we have we have intuitive views of almost everything. So as soon as you present a problem to me, you know, I some ready made answer and that gets in the way of clear thinking of those ready-Made answers, and we can't help but have them. So that's one thing that gets in the way emotions get in the way. I would say that independent clear thinking is. The first approximation, impossible mean in the sense that, you know, we believe in things most of the time, not because we have good reasons to believe that if you ask me for reasons that explain you and I'll always find a reason that the reasons are not the causes of our beliefs, we have beliefs because mostly we believe in some people and we trust them and we adopt their beliefs so we don't reach our beliefs.

But clear thinking something, you know, unless you are a scientist doing something like that.

But even then, it's probably a very narrow and that's very narrow.

And there is a fair amount of emotion, neuroscientist as well, that gets in the way of clear thinking, you know, commitments to your previous views, being insulted that somebody thinks he's smarter than you. I mean, lots of things get in the way even when you're a scientist. So I think there is less clear thinking than people like to think.

Is there anything that we can do at the belief formation stage? Like it sounds almost as though when you say that we're reading a newspaper, we read this op ed and it's well constructed and fits with our view of the world. Therefore, we adopt that opinion and we forget the context that we didn't learn it through our own experience or reflection. We learned it sort of from somebody else. So we don't know when it's sort of likely to work or not work, but we just proffer that as our opinion is there.

That's how I believe in climate change.

You know, I believe in the people who tell me there is climate change and the people who don't believe in climate change, they believe in other people.

So but similarly, there's like fake news and all this other stuff that we have the same reaction to you, you know, but but much more likely to believe fake news on my side than the fake news on the other side.

I mean, it's true that there is the huge degradation in public discourse and the recent 10, 15 years in the United States. I mean, there used to be an idea that facts matter.

What would be your hypotheses as to why that that is playing it without getting into politics? Because I don't want to talk politics. But why is that?

Well, I mean, it's hard to it's hard to answer that question without without politics, because at the general political polarization, I had a very big effect. And the fact that people can choose the sources of information.

Let's switch gears a little bit and talk about intuition. I think one of the things that strikes me the most about some of the work that you've done is the cases where we're likely to trust our intuition and when we're not.

And so if I'm correct me if I'm getting this wrong. So it's sort of like a stable environment, repeated attempts and rapid feedback.

It strikes me that most decisions made in organizations do not fit that environment. And yet we're making a lot of these decisions on judgment or experience. What are the ways that we can sort of make better decisions with that in the context?

Well, in the first place? I think, you know, you shouldn't expect too much talk to lower expectations. You should have low expectations about improving the systems. I mean, there is one basic rule is slow down, especially if you if you have the immediate conviction, slow down. There are procedures. You know, there are ways of reaching better, better decisions, but reaching better judgments. And we can talk about them. I would love to if you really want to improve the quality of decision making, use algorithms wherever you can.

If you can replace judgments by by rules and algorithms, they'll do better. Those big social costs to TRUSTe, allowing algorithm to make decisions. But but the decisions are likely to be better. So that's what if you can't use the algorithms, then you slow yourself down. And then there are things that you can do for certain types of problems and the different types of problems. So one class of problems forecasting problems. A friend, Phil Tetlock, in that book, Super Forecasters, where he identifies with people who are good at forecasting the future, but they do that makes them good and and tries to train people and they can improve.

So that's one classic problem. I'm into the. Specifically, in another kind of problem, judgment problems, where basically you're considering options or you're evaluating the situation and you're trying to give it a score, there, there there is advice, I think, on how to do it. For me, it goes back to something I did and the Israeli army when I was like 22 years old. So that's a long time ago. Like 63 years ago, I was a psychologist in the Israeli army and I was assigned the job of setting up an interviewing system for for the army.

That ridiculous that, you know, this was the beginning of the state of Israel. So people were improvising all over the place. I had a B.A. and I think thinking was the best trained psychologist in the on my my boss was a chemist. Brilliant. But anyway, and the existing system was one where people would interview and try to form an intuitive global image of how well that recruits would do as a combat soldier with the objective of the object of the interview.

And because I had read a book, I mean, I took a different tack and the different that was identified six traits that I sort of made up. And I had them ask questions and evaluate each of these traits independently and score it and write down the score, then go on to the next trip. And they had to do it for all six traits. That was that's all I asked them to do. And the interviewers who. But when you're younger than I am or recruits but very, very smart, selected for being good.

And they were furious with me and they were furious with me because they wanted to exercise their intuition. And I said, remember that one of them said, you are turning us into robots. So I compromised with them. And I said, OK, could you do it my way? And I told them, you try to be reliable, not valid. You know, I'm in charge of the ability to be reliable, which was pretty arrogant, but that's that's how I presented it.

But then when you're done, close your eyes and just put down a number of how good a soldier is that likely to be. And when we validated the results of the interview, it was a big improvement, what had gone on before. But the other surprise was that the final intuitive judgments. And it was good, it was as good as the average of the six, straight and not the same, it added information. So actually we ended up with a score that was half was determined by the specific ratings and the intuition that half the way and and that, by the way, stayed in the Israeli army for well over 50 years.

I don't know whether I think it probably some version of it was still in force. But around 15 years ago, I visited my old base and and the commanding officer of the research unit was telling me how they run the interview. And then she said, and then we tell them, close your eyes. So that that had stayed for 50 years, not the close your eyes. And that whole idea is now the basis of the book that I'm writing.

So I actually have the same idea, really, that when you are making decisions, you should think of options as if they were candidates. So you should make or break it up into dimensions, evaluate each dimension separately, then look at the profile. And and the key is delay your intuition. Don't try to form an intuition quickly, which is what we normally do. Focus on the separate points. And then when you have the whole profile, then you can have an intuition and it's going to be better because people make form intuitions too quickly and the rapid intuitions are not particularly good.

So if you delay intuition until you have more information, it's going to be better. I'm curious how we delay intuition, your intuition, by focusing on the separate problems. So our advice is that if you have, you know, the board of directors making decisions about the investment, we told them to do it that way, take the separate dimensions and really think about each dimension separately and independently. And don't allow me. And I feel the chair don't allow people to give the final judgment.

So we wait until we cover the whole of the. I mean, if you find a deal breaker, then you stop.

But if you haven't found a deal breaker way to the end and look at the profile and then your decision is almost certainly going to be better, does that include weighting the different aspects of the problem differently or do you highlight that in advance?

Or do you just I mean, it it makes you see the trade offs more clearly. Otherwise, when we don't follow that discipline, there is a way in which people form impressions very quickly form an impression, and then you spend most of your time confirming it instead of collecting evidence. And so if accidentally your impression was in the wrong direction, you're going to confirm it. And you don't give yourself a chance to correct the independence of the key, because otherwise, when you don't take those precautions, it's like having a bunch of witnesses to some crimes and allowing those witnesses to talk to each other.

They're going to be less valuable if you're interested in the truth than keeping them rigidly separate and collecting what they have to say. What have you seen work in a repeatable way? It may be a particular organization or cross organizations to not only reliably surface disconfirming evidence, but then place a value on what is surface instead of being dismissive. Is there a framework for that? Is there?

Well, there are many. You know them. There are many procedures like Red Team, Blue Team, Devil's Advocate members have been, you know, many attempts. And in general, you know, if you are if you're the head of a group that makes decisions, one of your missions would be to protect the dissenters. Because they're very valuable and you should make it painless to them as painless as possible, because it's hard to decide it's painful and costly, so protecting dissenters is important.

I'm curious about the distinction between intuition and judgment. You had mentioned intuition and judgment, intuitive judgment. Can you walk me through some of, like, how those differ? It's a bit hard to separate in judgment. This is what you do when you integrate a lot of information informally into the score of some kind. I we speak we being my co-authors in the book we're writing as we speak of judgment is measurements, but it's measurement with the measuring instrument is your mind.

But you do it informally. And because you do it informally, people are going to are not necessarily going to agree. So wherever we say it's a matter for judgment, we're allowing for differences, for variability. Now, judgment can be more or less slow, more or less systematic. So at one end, you have your intuition where you allow the judgment to go very quickly and so on. And at the other end, you try to delay intuition.

But ultimately, if you're making it by judgment, you're going to have a judgment and it's going to be like an intuition and you're going to go with it. So the more or less deliberate judgment, intuition is always involved at one point or another. You're sort of like listening to it or fending it off. Yeah, and our recommendation is turn it off. Are there ways to judge the quality of somebody's judgment? Oh, sure. I mean, some of them would be unique to the actual scenario, but what are the sort of other ways that we could?

Well, I mean, you you may require people to explain their judgments. And evaluating the quality of the explanation is, you know, whether it's logical, whether it's use of the evidence, whether it uses all the evidence, whether it is strongly influenced by wishes. Whether the conclusion was reached before the judgment supposedly is made, you know, there are a lot of there are lots of ways for them to fail that can be recognized. So it's harder to recognize very good judgment, but it's easy to see, you know, what goes wrong.

And there were quite a few ways to go wrong.

And I think some of those ways are the cognitive biases like overconfidence and sort of using small or extrapolating from small sample sizes. And one of the interesting things that I've heard you say in interviews before, correct me if I'm I'm off here, is that you've studied cognitive biases effectively your whole life, and you're no better at avoiding them than anybody else. Yeah, certainly not much better. Know what hope to the rest of us have? Not much. I mean, I never you know, I I think, you know, the quality of people's judgment is affected by my education, but.

So in general, you know, more educated people make better judgments, I think I'm average, but people decide I'm going to make better judgments. I don't think that's very hopeful. I'm much more hopeful about organizations because organizations think more slowly and they have procedures for thinking. And so you can control the procedures. Individual judgment is really hard to fix, not impossible.

One of the things that I see people do in response to cognitive biases and trying to account for them is to sort of make a list of them almost like a checklist and then go through that checklist and explain or rationalize why those things don't apply in this situation. It also strikes me that the more intelligent you are, the more stories you'd be able to conjure up about why why you're avoiding this.

I really think that's not very helpful because there are so many biases and the biases work in different directions anyway. So sometimes you can recognize a situation as one in which, you know, you're likely to be wrong in a particular way. So that's like illusions. If you if you recognize a particular pattern of something that gives rise to visual illusion, then you don't trust your eyes. You know, you do something else. And the same thing happens when you recognize this is a situation where I'm likely to make an error.

So sometimes you can recognize the importance, for example, of what we've called an anchor. So you're going to negotiate a price with somebody. They start very high and that has an effect. So, you know, or you should know the person who moves first and the negotiation has an advantage. Because it's the first no changes everybody's view of what is considered plausible to move things in that direction. That's that's a phenomenon. People can learn that and they can learn to resist it.

So when I was teaching negotiations, I would say somebody does that to you, comes up with the number, that's absurd. I would say lose your temper, make a scene, say I will not start the conversation from that. No, it's an absurd number. I don't want to let the rest of them. So that's something that they, you know, you can improve if you recognize. I think people are aware of the fact that you shouldn't make the decision about road safety within a short interval of a terrible accident.

And so you should allow things to settle down and cool down. There is a more subtle error and the harder harder to fix, but that the best prediction, the best guess is always less extreme than your impression. Intuitive prediction is, as we say, not regressive. It doesn't recognize the reversion to the mean, but statistics of statistics and statistics. Things are less extreme. Should I give you my favorite example of a bias? Yeah, I have been unable to think of a better one, but the story is about Julians.

That's part of the story. That's her name. She is a graduating senior at university. And I'll tell you one fact about her that she read fluently when she was four. But the GPA and and the interesting thing here is that everybody has a number. As soon as I told you that the number came to nine. Now we know where that number came from. We really that's one of the few things that I'm reasonably sure I understand perfectly. And this is that when you hear she read fluently at age four, you get an impression of how small tree of how precocious she was the day for.

And you could put that in percentiles, you know, whether that put her on a percentile for sort of aptitude ability and it's high, it's not, you know, if you read fluently at age two and a half would be more extreme, but it falls pretty high. So said the 90th percentile. And and then the GPA that comes to your mind is around the 90th percentile in the distribution of GPA. So you pick something. Your prediction is as extreme as your impression.

Mm hmm. And it's idiotic, statistically completely stupid, because clearly the age at which I learn to read is not all that diagnostic with respect to GPA. So it's better than nothing if you didn't know anything, you would predict the mean GPA, whatever they did, three point one, three point two. Now she's bright, so probably a little higher, but not three point seven. You don't want to. So that's cool. That's a bias.

That's not regressive prediction. And that's very hard to resist. Sometimes I'm able to resist it, but never when it's important. You know, when I'm really involved in something, I don't think about it. But sometimes I will recommend, oh, you know, that's a situation I should moderate my prediction. And if you're conscious of it, that's an example of one. You can sort of talk yourself.

Yeah, you can talk yourself into, although, you know, you usually will find a way to cheat and end up with your intuition. It's that's remarkable, you know, when you've been in academic life for a long time. So you've been in many situations where people discuss the job candidate and absurdities of that kind of very common. So somebody, a job candidate, gives a talk and people evaluate the talk and. Something happened, you know, at Berkeley when I was teaching there that somebody the was a very good dog in the bed now that of had teaching prizes.

And yet what was said about him in the discussion, he can't teach. You know, we heard the talk. So that's a mistake. But the funny thing is you can point out to people that that's a mistake. They still don't want to hire him because they gave a lousy talk. So it's hard to resist. It's interesting.

And one of the I think one of the ways I probably got my job is using psychology in the interview, which is asking why I was there and then reinforcing those beliefs throughout the interview.

I want to come back just one second to the immediacy of sort of having a stimulus and then making a decision. So we use the example of roads and a tragic accident happens. And you're rethinking sort of policy or laws or on the roads. How much of that do you think is social pressure?

And I'm wondering if we could even extrapolate that a little more to we're taught to answer questions on a test right away. Right. So we see this question and we answer it. We're taught that we or maybe it's reinforced taught us probably the wrong word, that politicians need to have a response, an immediate response to and even if they know the best thing to do is like, OK, like what they said, I'll take some time. It's society writ large seems to demand it like the environment is not conducive.

I think it's pretty clear that people prefer leaders who are intuitive and who are overconfident, leaders who deliberate too much are viewed with suspicion.

So I think Obama was at a certain disadvantage relative to George Bush, you know, because he was seen as more deliberate, it was all deliberate.

And then when you're very deliberate, you look as if you don't know what you're doing, but when you act with confidence.

So people want leaders who are intuitive, I think, by and large, provided they agree with just work my way back through some of these rabbit holes that we've gone down.

You taught negotiations. I'm curious what would be in your your sort of syllabus for negotiations that everybody should learn about negotiations when it comes to your work and psychology?

Well, you know, that goes back to a theme that we started with. The essence of teaching negotiations is the negotiations is not about trying to convince the other guy. It's about trying to understand them. So, again, it's slowing yourself down. It's not doing what comes naturally because trying to convince them is applying pressure. Arguments, promises and threats are always applying pressure. And what you really want is understand, you know, what you can do to make it easy for them to move your way.

Very nonintuitive. That's a surprising thing. When you deep negotiation, it's not obvious. You know, we are we're taught to apply pressure and socialize that way.

You mentioned that there is procedures for thinking in organizations. Are there any that stand out in your mind that we could use to elevate thinking and if not elevate but give feedback on the quality of thinking to improve it? Well, I think one of the ideas that people like the most is when I did by Gary Klein and he calls the premortem. And that's that's universal when people really like that idea. And this is that when you are about to make a decision, a group.

Not quite, because if you've made it, it's too late, but they're approaching you. And then you get people in the room who can be the people who are making the decision. And is that supposed it's two years from now we made the decision that we're contemplating and it turned out to be a disaster. Now we have a page in front of you write the history of the disaster, and that's the premortem. And and it's beautiful as an idea.

It's beautiful because when people are coming close to a decision, it becomes difficult to raise doubts or to raise questions. People who are following the group down when the group is nearing a decision are perceived as really, you know, and really, you know this. They want to get rid of them. And the premortem legitimises that sort of dissent and that sort of. Not only legitimizes that, but rewards it, and so that's a very good idea.

I don't you know, I don't think that it's going to prevent people from making mistakes, big mistakes. But it could certainly it will allow people to possible loopholes to to things that they ought to do to make us safer. So that's a that's a good procedure. And there are many others. What comes to mind? What comes to mind is, is to make intelligence and the collection of information independent of the decision makers wishes. And you really want to protect the independence of the people collecting the evidence.

And I would add to the procedures, really, people don't like that if it were possible to implement it, I think would be good. And that's. Look, when you are going to be discussing the topic and it's known in advance and people in central material to think about the topic that you may want them to write down their decision, the decision they are in favor of before the discussion starts, that has many advantages. It's going to give you a broader diversity of points of view because people tend to converge very quickly in a in a group discussion and it forces people to be better prepared.

So it's except people don't want this. So I don't know whether it's even possible to implement. But clearly, if you could be a good idea, what are the reasons people don't want to do much work it forces you to do rather than the signaling you can sort of get away with? Yeah.

And then, you know, somebody who is going to prepare the case. And so I glanced at the material and then, you know, so a lot of meetings are tremendous to think for wasted time and improving the quality of meetings would be a big thing. Do you have any insights on how to do that, keeping them short? You know, I'm not a professional at fixing meetings, so I have I have a few ideas, but not a complete view.

The the question of structuring the meetings to be discussing topics one at a time that I think is is really useful. I'll give you an example. I mean, it's something that I suggested when I was consulting, but for some reason, people didn't buy that suggestion. So you when an investment is being discussed, so by the investment firm, some staff people, if it's a big investment stuff, people will prepare a briefing book with chapters. Now, our recommendation would be that the staff should end each chapter with a score.

How does that chapter, taken on its own, independently of anything else, affect the likely decision? And then you could structure the meeting, the discussions and the meeting of the boards to discuss these schools. One of the things that has the effect that I was talking about earlier, making the decision, making the judgments about the dimensions we call them mediating assessments is all drawn to the mediating assessments. Come first and then you have the profile of them, and then you make a global judgment and you can structure it.

So if the staff has presented a score and you discuss in the board, do we accept the school, you're forcing people to have a look at the evidence and think about why they would accept or reject.

And then they feel like you have to construct an argument that might be less intuitive, etc..

So, you know, there are ways of doing this, but if you're going to be too rigid about it, it won't work either.

So I'm curious what other advice you gave as a consultant that nobody followed?

I mean, virtually all the advice we get, people don't follow. I mean, you know, I think that that's not you shouldn't you know, you're not going to be a consultant. If you expect your advice to be taken. You have to give the best advice you can.

What would be other examples of something you think could be widely applicable that you would advise you would have advised people and you just sort of like saw them drop the ball?

Well, I mean, you know, I would advise people who make a lot of decisions to keep track their decisions and of how they turned out so that later you can come and and evaluate your procedures and and and see whether there is anything that is in common with those decisions that turned out well and then not so well and so on. People hate doing this. Why do you think people hate doing it? Oh, because because retrospectively they may look foolish, some of them or all of them or in particular the leader.

So they really don't like it. I mean, there are exceptions, Ray D'Alessio and his firm where everything is Bridgewater's. Yeah, yeah. Redwater but in general, it makes having consulted with Bridgewater, they don't need me. But but in general, when I suggested that never went anywhere, what are the variables that you would recommend people keep track of?

Like what would your decision journal look like? Oh, you know, my my decision journal would be a mess. I don't know, putting myself as an example, but so obviously the outcome, but you've got to do that post after all.

But no, not you. You would want to say what were the main arguments, pro and con? What were the alternatives that were considered? And it doesn't have to be very detailed, but it should be enough so that you can come later and debrief yourself.

Should you have a calibration like what degree of confidence you are that would be good.

Then, you know, it would depend on something that you could evaluate.

It strikes me that decision journals and premortem are a way to identify people that are sort of perhaps suppressed by their manager, where you have somebody who's actually a better, better at exercising judgment than the person that is that they're working for. And this would be a pain free sort of way to calibrate that score over time and identify the quality of judgment in a consistent way. We are. And that strikes me as where a lot of money to an organization.

Yeah, but they're also very costly. And you you will see that certainly anything that threatens the leader is not going to be adopted. And and leaders may not want something that threatened their subordinates, whether people are really very worried about embarrassment.

You're writing a book now on noise. Yeah. Tell me about Noise and Decision-Making. Can you explain the concept?

Yeah, I can really explain it by saying what you know was the beginning of it, which was a consulting assignment and an insurance company where we had the idea of running a test to see whether people in a given role were supposed to be interchangeable. Agreed with each other. So, you know, when you come to an insurance company and an underwriter gives you a premium, the underwriter speaks for the company. And so it's you expect that any underwriters.

It doesn't matter which underwriter you get to for the premiums and the company has that expectation shouldn't make much difference. So we tested that and they constructed some cases. And then we have some like 50 underwriters and a premium for the case with the same information.

Yeah, with a really very realistic we didn't construct they constructed the case, so they conducted the experiment. But now the interesting question is how much variation do you expect there to be? So we asked the executives the following question supposedly to take two underwriters, referendum them by what percentage do they differ? You look at the difference between the premium divide, that by the average premium with no judge and people expect 10 percent. By the way, it's not only the executives in that company.

For some reason, people expect 10 percent and it was roughly 50 percent, five zero. So that's you know, that's what made me curious about that. And the fact that the company was completely unaware that it had most took them completely by surprise. So now we're writing a book because there's a lot of noise. So wherever a rule is that wherever there is judgment, there is noise and more of it than you think. So that's the pattern.

Are there procedures to reduce noise? And conversely, is noise? It strikes me that the variation would be good, but maybe only in an evolutionary concept.

Well, we call that noise is useless variability. And variability can be very useful if you have a selection mechanism and some feedback. So evolution has built up variability, but of course it's useful. But noise among underwriter's is useless. There's nothing nothing gets learned. There's no feedback. It's just noise and it's costly. The first advice, of course, would be algorithms, as I said earlier. So algorithms are better than people and judgments. That's not intuitive, but it's really it's really true.

And and after that, then, you know, the procedure that I mentioned earlier for making decisions in an orderly way by breaking it up into assessments. And that's the best that we can do. And and there is one very important aspect that I haven't mentioned. But and this is training people in that the scale is so there is one piece of advice that you'd have for underwriters that they should always compare the case to other cases. And if possible, if you can have them share the same frame of reference with other underwritings, we're going to cut down on the.

Oh, that's a clever idea, controlling the scale and that exists in human resources, where performance evaluation, which is one of the scandals of modern commerce, how difficult it is, but performance evaluation, they have the thing that's called frame of reference training. Which is teaching people, you know, how to use the scale, there's a lot of variability in the scale and part of what the super forecasters do, they make judgments in probability units and they teach them to use the probability scale.

So learning the scale is a very important aspect of reducing noise.

I know we're we're coming up to the end of our time here. What have you changed your mind on in the past 10 years? Oh, lots.

And I think that, yeah, there's been a replication process in psychology. And some of the stuff that I really believed in when I wrote Thinking Fast and slow, some of that evidence has been discredited. So I've had to change my mind. What are the what's the some of the facts, this stuff priming and unconscious priming and so just hasn't held up and replication. And I believe that and I wrote it as if it were true because the evidence suggested it.

And in fact, I thought that you had to accept it because that was published evidence. And and I should have I blame myself for having been a bit gullible, that as I should have known, that you can publish things even if they're not true. But I just didn't think that through. So I changed my mind. I'm now much more cautious about spectacular findings. I mean, very recently of I think I have a theory about why psychologists are more social scientists generally are prone to exaggerate the to be overconfident about their hypotheses.

So I've done quite a bit of loanwords. What's the theory on the theory that one element of the theory is that all of these hypotheses are true? In what sense that you might find that there's a famous study that you mentioned wrinkles to people and then you measured the speed at which they walk and they walk more slowly. Turns out that hasn't held up in replication, which is very painful. It's one of the favourite studies. But actually, you know that if you mentioned Wrinkle and it's going to have any effect on the speed of walking, it's not making to me.

It's not going to make people faster. It has an influence to make them slower. So directionally, all these hypotheses are true. But what there is, is what people don't see is that the huge number of factors that determine the speed at which individuals walk and the differences in the speed of walking between individuals and that noise and people neglect noise. And then there is something else which is touches on both philosophy and psychology when you have intuitions about things, the clear intuitions and their strong intuitions.

Another thing so clear intuition is if I offer you a trip to Rome, a trip to Rome and an ice cream cone. You know what you prefer, it's easy, but it's very weak, of course, mean the amount of money you would pay to get a trip to Rome and a trip to Rome. And then I go nothing. But when you are a philosopher and I should add one thing, to see the clear intuitions, you have to be in this kind of situation that psychologists call within subject, that you have both you have both the with with the ice cream cone and without the ice cream, the.

So, you know, within subject situation, that's an easy problem in a between a separate situation, it's an impossible problem. But now if you're a philosopher, you're always within subject situation. And but people live in between subjects that you wish they live, you know, in one's condition. And the same thing is true for psychologists. So psychologists live in when they cook up their hypotheses, they're within a subject situation. But then they make guesses about what will happen between subjects and they're completely lost between clear intuitions and strong intuition.

We have no way of calibrating ourselves. So that makes us wildly overconfident about what we know and reluctant to accept that we may be wrong of. That's a great place to end this conversation. Anything you saw on. The knowledge project is produced in collaboration with Jason Oberholtzer and the team at Charts and Leisure. You can find show notes on this episode as well as every other episode at F-stop blog podcast. If you find this episode valuable, shared on social media and leave a review to support the podcast, go to F-stop logged membership and join our learning community.

You'll get hand edited transcripts of all the podcasts and so much more. Thank you for listening.