I’ve got this friend and neighbor who I hike walk with after we drop the kids off at school sometimes. It’s like 90 minutes of dorm-room musings befitting of the Moontower scene. Those conversations have influenced a few of these weekly letters.
This recent one was no different. It got me thinking about diversity.
See, this friend has an early-stage startup in the education space. The internal research at his company parses diversity across many dimensions. You are familiar with the capital “D” types of diversity — race, religion, gender, age, sexual orientation, socioeconomic, etc.
Interestingly, he expressed concern that his team might not be diverse enough. Not in a visible way. The team runs the gamut of the capital “D” diversity categories. But he was interested in cognitive diversity. He was concerned that an intellectual echo chamber of fancy-college liberals could lead to blind spots in their collective decision-making.
Now I don’t have much team-building experience. So I flexed some knowledge I recently read. (This is why people read right? To at least have a tennis racket when they find themselves on the court of conversational Wimbledon.) And now you too shall witness the fact that I read a book. Confer prestige heavily and without reservation, my esteemed landsmen.
Nah, really I actually read a book and it said some cool stuff. I’ll get to that in a sec. First, I want to re-surface some points professor Mauboussin made about cognitive diversity. He described it as the training, experience, and personality that make an individual unique.
He writes:
I think one can make the case very seriously and quite rigorously that social category diversity contributes to cognitive diversity, but it is cognitive diversity that we’re after.
He describes a less-talked-about form of diversity as well.
“Values diversity”. You might think about it as a sense of purpose, and on that, you actually want to be low. We want a common mission, even if we are of very different backgrounds, we’re pulling in the same direction.
In other words, my friend’s instincts about diversity are correct. Visible diversity is an imperfect proxy for intellectual diversity.
Back to the book I was reading — Superforecasting by Phil Tetlock and Dan Gardner. The book’s main thesis, which falls out of the lessons from the Good Judgement Project, is that it’s possible to become a well-calibrated forecaster in complex (but not all complex) domains. How good can you become? The best are able to consistently beat prediction markets, something even demonstrably above-average forecasters struggle to do.
This is rightly provocative because markets are effective truth-finding mechanisms. They are an ancient way of coordinating human behavior (democracy is another example of a human-coordination machine…for a contrast between markets and democracy see Dinosaur Markets).
If you read this letter regularly you know I have a lot of respect for the efficiency of markets. It’s not a strong-form academic belief. It’s more of an informal razor: “my null hypothesis is there’s no easy money and the burden of proof is on investors who think otherwise”. The academic compromise is markets are “efficiently inefficient”, reflecting the idea that there’s a cost associated with finding inefficiencies so some amount of inefficiency always exists to justify the hurdle of hunting for it.
To appreciate why markets, under certain conditions, triangulate on the truth, I paraphrase Tetlock’s explanation of how the “wisdom of crowds” works:
Bits of useful and useless information are distributed throughout a crowd. The useful information all points to a reasonably accurate consensus while the useless information sometimes overshoots and sometime undershoots but critically…cancels out.
The Role Of Diversity In Truth-Finding
The “under certain conditions” is an important asterisk. The expression “wisdom of crowds” is actually a modern idea that plays off the “madness of crowds”, a term coined nearly 200 years ago by journalist Charles Mackay. For the crowd to generate wisdom, it needs diversity. In other words, the errors in judgment need to be uncorrelated to cancel out.
In Tetlock’s studies, they tested the forecasting abilities of individuals. They were rigorous in their experimental design. They were curious how teams of forecasters would perform against individuals. They further experimented with the composition of those teams.
The eye-opening results underscore the importance of diverse thinking:
- Teams are more effective
- The results were clear-cut each year. Teams of ordinary forecasters beat the wisdom of the crowd by about 10%. Prediction markets beat ordinary teams by about 20%. And superteams beat prediction markets by 15% to 30%.
- “Emergence”: teams are more than the sum of their parts. This cuts both ways…even actively open-minded individuals could surrender to “groupthink”
- “Diversity trumps ability”
- This provocative claim highlights how the aggregation of different perspectives can improve judgment. The key to diversity was, unsurprisingly, cognitive diversity.
- The revealing result: When they constructed the superteams they optimized for ability and those teams happened to be highly diverse because the superforecasters themselves were highly diverse. They did not optimize for diversity first, but it turned out the most diverse teams were the most effective.
- This provocative claim highlights how the aggregation of different perspectives can improve judgment. The key to diversity was, unsurprisingly, cognitive diversity.
- The asymmetry of the extremizing algorithm
- The “extremizing algorithm” is a technique where you boost a 70% prediction closer to the extreme, perhaps bumping it to 85%. It’s a technique that is employed when the forecasters have diverse perspectives because it leads to better-calibrated forecasts.
You do the opposite (push the forecast probability closer to 50%) to combat “groupthink” if the team is comprised of people who think the same or possess similar knowledge. (The use of the extremizing algo allowed teams of regular forecasters to actually perform better than some superteams!).
My own observation: this is the same logic by which correlated observations “shrink” the sample size, an idea familiar to data analysts.
- The “extremizing algorithm” is a technique where you boost a 70% prediction closer to the extreme, perhaps bumping it to 85%. It’s a technique that is employed when the forecasters have diverse perspectives because it leads to better-calibrated forecasts.
Example Of Cognitive Diversity
My friend with the start-up gets it. He is concerned that the visible diversity on his team might be a poor proxy for what he really wants — cognitive diversity. If you are an oil company you need geologists, finance people, managers, political connections, real estate expertise. This is a clear example of needing to pull together many object-level competencies.
But cognitive diversity is not just “what do they think about”, but “how do they think”. For an example of what I call meta-cognitive diversity, Tetlock uses the “hedgehog” vs “fox” duality. As a pre-defense of Tetlock, he warns about overstating this dichotomy. (He exemplifies non-binary thinking throughout the book, doing an honest and eloquent job of pointing out tensions and caveats, not unlike the superforecasters themselves).
I’ll take a stab at describing hedgehogs and foxes:
- Hedgehogs
Hedgehogs are specialists. The 10,000 hours crowd. The natural endpoint for the logic of economic comparative advantage or simply the rightful throne of the devoted craftsman. The “specialization is for insects” objection is too reductive. We are better off when Eddie Van Halen wants nothing else than to just be Eddie Van Halen.
But there are trade-offs. Hedgehogs often filter observations through the lens of their expertise. (My online friends have a good-natured running joke that I see everything as an option — classic “when all you have is a hammer, everything is a nail” thinking). But a camera lens’ usefulness depends on the context. Sometimes that fisheye or telephoto lens is exactly the wrong tool for the job.
This is inconvenient for the status-aware hedgehog whose incentive to remain consequential leads to motivated reasoning and self-delusion. Academic researchers who become famous for writing about an idea that catches fire have a lot to protect. They become fast friends with Mssr. Confirmation Bias and Madame Overconfidence, the very enemies they used to fight when they were building their reputations of good work. It’s like America fighting against the same Afghans they armed as rebels in the 80s. As DiCaprio’s character in Up In The Air would attest, the warm embrace of fame beats the cold loneliness of cultural anonymity.
- Foxes
Foxes are generalists. Businesspeople, investors, politicians, administrators. It’s an imperfect description of course but you know the type. The disadvantages of being an “inch deep and a mile wide” are established. That person is never going to design a bridge or coach a professional sports team. Some are self-aware enough to recognize when they “know enough to be dangerous”. Many are not. Their advantage, however, is their mercenary relationships with lenses. The ego cost of finding the most useful lens is much lower, making foxes at least psychologically fit for reasoning across domains.
Remaining careful not to play into false binaries, the hedghog/fox continuum reminds me of the importance of shifting between diffuse and focused thinking modes. Diffuse thinking (see More Shower Thoughts Please) provides both inspiration and the ability “see over the neighbor’s fence” while focused thinking enables us to synthesize those scattered insights into a useful output. Most of my posts start either in the shower or when I’m taking a walk. (I strongly recommend this old New Yorker piece Why Walking Helps Us Think). “How we think” is a dimension of diversity.
The Complicated Discourse Around Diversity Is Inevitable
When I described the thrust of this piece to my wife, Yinh, she yawned. “So you’re writing a post saying ‘diversity is good’? Who is this news to?”
I immediately got nervous that I was belaboring an obvious point. I asked her why she thought it was so obvious and she started citing studies and initiatives that could easily have 10x’d the length of this post.
Now I can take feedback, but I’m not above quibbling en route to my final destination. So let me get this straight. You read a bunch of stuff arguing that diversity is good, now it’s obvious to you and presumably everyone else who reads, so I shouldn’t spend any time making arguments that diversity is good. I mean, this post is to a Moontower reader as that other research was to you. Next time just put my head in the washing machine woman.
But she still has a point. Diversity, in its many forms, is widely celebrated. In-breeding is taboo. I was just watching the National Parks series on Netflix with the kids and learned that rainforests, the most biologically diverse ecology on land, are the origin of more than 25% of modern medicines. So why do I feel the need to cheerlead an “obvious” point?
The short answer is I don’t think the case is closed on the merit of diversity. I know it’s exhausting to hear me say this, but it depends on contexts. For example, whenever America’s dysfunction is compared to a homogeneous European country, I scratch my head. If we are tribal in nature, our default wiring might simply make governing a melting pot inherently more difficult. I think diversity makes us stronger overall, but some measures of local harmony should expect to suffer. I’m even open to the possibility that tolerance runs counter to our natural instincts. (It just makes a normative approach to overriding our base impulses require extra care. Law-making is always a tug-of-war between collective values and the animals within us.) In other words, I can appreciate how diversity can be a headwind.
But there’s more.
Even if we wave a wand and agree that diversity is an unalloyed good, there remains the harder question. At what cost? The lightning rod version of this question is you have a white student and black student who look the same in all other ways and you need to choose one (the premise is unrealistic, but this is the collapsed version of how these questions get passed around the media and people extrapolate entire political identities on how they’d answer such a fake question). If everything else is the same and there is a non-zero probability that social diversity leads to cognitive diversity, then the optimal (although not necessarily morally fair which is a different criteria battleground) decision gate would say select the black kid (assuming the majority of the student body were white).
Still, even if we agreed on that, a harder question remains. What if the black kid had a slightly worse score on a standardized test? From a strictly efficient-utilitarian point of view (again, moral consideration aside), then we are faced with trade-off on a diversity-competence frontier. In a purely academic sense, I was likely an inferior hire at SIG, but perhaps something about how I thought or acted might have made my “diversity” or “complementariness” worth more than just hiring yet another 1600 Math SAT kid from MIT (or they just exhausted the supply of those, I’m not trying to flatter myself here). The point is that the merit of diversity is fairly intuitive, but doesn’t lend itself to legible number-crunching in the way test scores do.
Pricing “diversity” could very well be a fool’s errand. But I have one final bit of intuition to sprinkle on the problem. In There’s Gold In Them Thar Tails: Part 2, I rehash how nature uses diversity as fuel for evolution.
- Diversity is an essential input to progress. Nature’s underlying algorithm of evolution penalizes in-breeding.
- In addition to a loss of diversity, signals decay as you get closer to the extremes. This is known as tail divergence. The signal can even flip (ie Berkson’s Paradox).
- The point where the signal noise overwhelms the variance in the candidates is an efficient cutoff. Beyond that threshold, selectors should think more creatively than “just raise the bar”.
At some point, incremental diversity is worth more than incremental “signal”. Evolution acts like a basket of options (I really am a hedgehog). It sees a mutation. If it’s useless, discard. If it’s adaptive, exercise it and let it multiply through the population out-competing those without the adaptation. When discernment becomes random, select for diversity. The downside is limited, the upside is massive learning.
Wrapping Up
Diversity is valuable. The word is highly politicized today. There are many arguments, with varying degrees of merit, harping on diversity in the name of fairness. Those are debates that need to be had. But this is not that debate. This has been an argument that diversity is important as a matter of efficiency and flourishing. Perhaps the argument isn’t needed, but I suspect that many reactionary arguments against the equity angles of diversity may dilute the value of diversity as a general concept.
We have seen diverse dimensions all around us: social, cognitive, and values themselves. It’s worth unloading the hangups artificially narrowing the meaning of a beautiful word — “diverse”.