The devil’s trick — making you believe incentives are everything

Charlie Munger once said “show me the incentives and I’ll show you the outcome”. Munger is a paragon of wisdom. Just this week I gave one of my son’s besties Charlie Munger’s Almanac for his 13th birthday.

But…there is a but.

As Adam Mastroianni starts in with his essay Startling differences between humans and jukeboxes:

The cause of every social problem, we can all agree, is that people get rewarded for doing the wrong things. Academic fraud, dysfunctional healthcare systems, good-for-nothin’ politicians—all cases of bad incentives.

I used to nod along to these conversations like yes, yes, of course, the incentives! But then I started paying attention and was like, wait, what are we all talking about?

I think there is a monstrous theory of human behavior lurking here—one that I myself have believed—that needs to be dragged out into the light and thoroughly stomped.

What follows is a remarkable essay that put its finger on something I’ve felt — incentives are far from the whole story when it comes to human behavior. The point of my bear on unicycle note is that sometimes our “why” is simply “because we can”.

But Adam closed the circle I couldn’t — Munger’s strong-form contention doesn’t hold (not shocking, there’s always exceptions) but the uneven effectiveness of incentives on people points to a far more complicated picture of humans. And that picture leads to a selection effect own goal (emphasis mine):

When you rejigger incentives in the hopes of changing behavior, you attract the people who are most motivated by the incentives themselves, and these are the people you want to attract the least. Incentive-hunters are bent on Goodharting you, that is, doing exactly what it takes to extract the reward, even at the expense of what you actually wanted them to do.

I always learn fascinating ideas and stories from Adam’s post. He likes to coin catchy terms for behavioral phenomena. This one is no different. Jukebox theory. Secret criminal theory. Find out what they are and more interestingly why they persist despite being the “wrong model of human behavior”.

This is a selection of quotes that stuck for me:

On training researchers:

Eventually I realized, no, what I really want are the students who give a hoot… It’s not that hard to give people skills. It’s way harder to give them interests… trying to substitute external incentives for internal incentives is like trying to power your country with whale blubber when everybody is walking around with a hydrogen cell battery inside them.

An obvious rebuttal to common rationalizations:

Anybody who’s like, “I hate lying to the Securities and Exchange Commission, but those are the incentives in finance!” or “I hate writing crappy papers, but those are the incentives in academia!” or “I hate making hamburgers that are mostly sawdust, but those are the incentives in the restaurant industry!”—they’re admitting that they’re only going to do the right thing when it’s convenient. I want to find the people who are willing to do the right thing even when it’s inconvenient, and hand them some money so they can keep doing that.

On a perverse inversion that slips by us too easily:

When people are like, “It’s too bad these bad incentives turn good people into bad people!” I’m like, “No, it’s too bad these bad incentives allow bad people to exist and succeed.” The good people are the ones who don’t turn bad even when it’s lucrative to do so. In other words:

That tweet is more strongly worded than its underlying veracity warrant but it holds enough truth to make you consider what we let slip by too easily. To accept Munger’s assertion uncritically is to pre-excuse trash behavior in a society-wide shrug of “don’t hate the player, hate the game”. Well, the game is made of players. If the well is poisoned because there were some incentives to poison it and we are somehow just ok with that because we treat outcomes predicted by incentives as laws of gravity then you’ve traded the virtue of norms for the consonance of devil logic.

I don’t know. The fact that this post landed so hard for me makes me think we are numb to how easily we sanewash sinners, worn down by an insidious premise that we are helplessly controlled by incentives. It’s a call to push back on this strong-form assumption that either doesn’t hold or when it does, it’s akin to shining a light on a jellyfish. A spineless jellyfish.

Fortunately, Adam ends with a sentiment that rocks the moontower spirit:

We act as if the improvement of humanity is an engineering problem, when really it’s an unleashing problem.

The only way to solve that problem is to climb out of our own boxes and to help other people climb out of theirs. That, of course, takes courage and trust.