Illusions, Empathy, and the Hackability of Perception

“Hey, what happened to the dinosaurs? Weren’t they just here?” Maybe comets killed the dinosaurs, maybe they tripped and fell. What’s the difference? We’ll never know. We couldn’t solve the Kennedy Assassination, we had films of that. Good luck with the Stegosaurus.”

–Jerry Seinfeld from his 1993 book SeinLanguage

As a teen, I read that book probably 20 times. My brain was full of Seinfeld takes thanks to the book basket next to our bathroom toilet. The book is still in that bathroom more than 25 years later.

Good luck with the stegosaurus

I remember having that exact reaction in the wake of the “dress illusion” that swept the internet nearly 5 years ago.  You can be staring at the same picture as your neighbor and disagree on the color. Then in 2018 the yanny/laurel debate exposed how different our audio perceptions are. These illusions are unsettling. If we can’t agree on what appears plainly before us, what hope is there for mutual understanding on issues.

It’s a cynical thought. And I’m not sure it’s wrong. The minimum amount of structural disagreement that would exist if everyone agreed on the same facts is disturbingly high. Call me naive, but this was a dark realization. And once you notice how futile striving for agreement is, the world does nothing to help you unsee it.

The Hackability of  Perception 

Scott Alexander writes:

“I’ve been focusing a lot lately on the idea of the Bayesian brain and its input channels. Some input channels, like vision, are high-bandwidth; we get so much data about the real world that we usually see pretty much what is really there.

Other channels, like pain, are low bandwidth. This is why the placebo effect works – we get so little data about how much pain is coming from different parts of our bodies that even our strongest percepts are wild guesses, where we fill in the gaps with predictions and smooth away conflicting evidence. If our predictions change – ie we know we just got morphine and morphine lowers pain – then the brain will happily change its guesses. This would never happen with vision – I can’t use the placebo effect to make you think an orange crayon is blue – but pain is low-bandwidth enough that it works.

Vision, with all of its flaws, represents a ceiling on perceptive fidelity. Sensations like pain are lower bandwidth. Vision is hackable but less so than pain. And unfortunately, logic and reason are even more hackable.

He continues:

Reason is one of the lowest-bandwidth channels of all, which is why biases are so omnipresent and rational debate so rarely changes anyone’s mind. Most people revert to their priors – the beliefs of their tribe or the ones that fit their common sense – and you have to provide an overwhelming amount of rational evidence before the brain notices anything amiss at all.” (Link)

And then the lesson appeared.

The Real Meaning of Empathy

I’ve often thought of empathy as being able to put yourself in another’s shoes. Commiseration for what you have not experienced but could imagine. Pain, loss, anxiety. We can relate to these feelings. But that is only half of what it means to be empathetic.


Because you presume that the other person’s sense of loss correlates with how you would feel in the same scenario. But if you can’t agree on what color the dress is, how can you know how Mary feels when she loses her dog? True empathy requires tremendous humility. You are not living their loss through your eyes but accepting that your eyes are irrelevant. In a world in which we simply will never agree, empathy that is contingent on being able to relate is an empty virtue.

These illusions teach us that sometimes we can’t relate. Empathy needs to be ok with that.

For the past few years, I have seen illusions as a heat check. They are as humbling as they are entertaining. Magic shows, mentalists, and illusions are a constant reminder that our perceptions are heuristics that can be exploited. And if our thoughts are built upon our perceptions, we should afford them leeway in proportion to how much error lives in what we see, hear, and remember. Humility is not optional if you value honesty.

Paul Graham has described our minds as a compiled program that we’ve lost the source code to. It works, but we don’t know why.

How Do We Protect Ourselves From Illusions

1) Recognize we are not natural fact-checkers. Be humble since you are usually not thorough. (Link)

2) Be aware of biases

Nick Maggiulli highlights:

  • The peak-end rule: we tend to focus on the most intense and the most recent parts of a memory when judging an experience. Professor Dan Ariely has pointed out that this flies in the face of conventional wisdom to rip band-aids off quickly. He was a survivor of severe burns and used this knowledge in advising nurses about the daily removal and redressing of his bandages.
  • The serial-position effect: People tend to remember the first things (primacy) and the last things (recency) in a list, but tend to forget the middle things. Nick cleverly demonstrates this with the following example:
    • You sluhod be albe to raed tihs snetnece tohguh amlsot ervey wrod is spleeld icorenlrcty.

3) Appreciate that your memory is not a video camera

Professor Dan Gilbert on our memories being edited and unreliable:

“We try to repeat those experiences that we remember with pleasure and pride, and we try to avoid repeating those that we remember with embarrassment and regret. The trouble is that we often don’t remember them correctly. Remembering an experience feels a lot like opening a drawer and retrieving a story that was filed away on the day it was written, but…that feeling is one of our brain’s most sophisticated illusions. Memory is not a dutiful scribe that keeps a complete transcript of our experiences, but a sophisticated editor that clips and saves key elements of and experience and then uses these elements to rewrite the story each time we ask to reread it.”
Taking advantage of illusions

The Placebo Effect

We already saw how the placebo effect can be used to trick the low-bandwidth input channel of pain. Dan Ariely has claimed that the “mind is so powerful as evidenced by the placebo effect. The link between expectations and beliefs is astounding.”

Scott Alexander (a psychiatrist by trade) recaps the explanation of the placebo effect from Surfing Uncertainty and when you would expect it to be most effective:

Perceiving the world directly at every moment is too computationally intensive, so instead the brain guesses what the world is like and uses perception to check and correct its guesses. In a high-bandwidth system like vision, guesses are corrected very quickly and you end up very accurate (except for weird things like ignoring when the word “the” is twice in a row, like it’s been several times in this paragraph already without you noticing). In a low-bandwidth system like pain perception, the original guess plays a pretty big role, with real perception only modulating it to a limited degree (consider phantom limb pain, where the brain guesses that an arm that isn’t there hurts, and nothing can convince it otherwise). Well, if you just saw a truck run over your foot, you have a pretty strong guess that you’re having foot pain. And if you just got a bunch of morphine, you have a pretty strong guess that your pain is better. The real sense-data can modulate it in a Bayesian way, but the sense-data is so noisy that it won’t be weighted highly enough to replace the guess completely. 

If this is true, the placebo should be strongest in subjective perceptions of conditions sent to the brain through low-bandwidth relays. That covers H&G’s pain and nausea. (Link)

An example from my recent trip to the Lawrence Hall of Science

Consider this statement again:

In a low-bandwidth system like pain perception, the original guess plays a pretty big role, with real perception only modulating it to a limited degree

In the virtual reality exhibit currently on display, there is a recording playing. I’ve reproduced it here. If you’d like to play along, have a friend silently mouth the word “far” in sync with the recording. If you look at them while they do this, that’s exactly what you’ll hear. If you look away, you’ll hear what I actually recorded. In the exhibit, they show a silent video with 2 people mouthing different words. As you flip your gaze between them you hear what they are mouthing although the recording is always saying the same word.

More fun illusions

Here are 2 optical mind benders I’ve collected over the years.

  • The cafe wall (Link)
  • Sinusoidal waves (Link)

Leave a Reply