Our brains are amazing, allowing us to do so much. Even so, they aren’t perfect. Do you remember during COVID, you would meet people for the first time wearing masks? Then, when you finally saw their whole face, you were surprised. And it’s not like you had purposefully thought about what the bottom half of their face looked like, but it was still somehow different than expected.
Facial Recognition & Pareidolia
This may be due in part to the fact that humans are wired to recognize faces. This is such a strong skill that we even recognize “faces” in inanimate objects: the front of cars, electrical outlets, even clouds or rocks. This phenomenon is called pareidolia, a human tendency to recognize meaningful patterns, especially faces. This is thought to evolve from the need to quickly recognize facial expressions for threats or social cues. We even have an entire region of the brain responsible: the face fusiform area (FFA) (Zhou & Meng, 2020). Studies have shown that even infants can recognize face-like patterns as faces (Kato & Megitani, 2015). Infants as young as seven months were able to recognize faces in Arcimboldo’s paintings (Kobayashi et al., 2012). These paintings were of inanimate objects, such as vegetables, arranged to resemble a portrait of a human. They are honestly a bit creepy, but feel free to give it a google!
Obviously we can all recognize a face, even under a mask, but a study using an artificial face recognition system gave us some evidence for why. A system called deep convolutional neural networks, or DCNN, determined the most important facial feature for facial recognition to be the eyebrows. This was followed by the eyes, mouth, then nose (Zhang et al., 2024). This makes sense with facial recognition under a mask, as the two most important features are still visible. Why the eyebrows are the most important isn’t clear. It could be that eyebrows are important for facial expressions, or that they are important for individual recognition due to their large diversity in color, shape, and placement.
Cognitive Dissonance & Bias
I found the science behind facial recognition fascinating, but it still doesn’t quite explain why we subconsciously fill in the bottom half of someone’s face. As it turns out, recognizing parts of a meaningful pattern, such as the face, but missing key parts of it, creates cognitive dissonance. Cognitive dissonance is when there is discrepancy between what your brain expects or believes and the information it is gathering. In the case of a mask, your brain is expecting to see all the important components of a face, so when the mouth and nose are missing, it creates a sort of tension. To relieve this tension, your brain uses what it knows about the face to fill in the gaps. This way, the pattern fits in with the categories we already have.
Significance
Pattern recognition, especially facial recognition, is super important, allowing us to recognize slight changes in facial expression or obstructed faces, like under a mask. But this can also lead to something not so great. Amy E. Herman said in her book “Visual Intelligence” that humans see through the brain, not the eyes. All your sensory organs collect signals and send them to the brain for processing. The signals don’t mean anything on their own, whether that is light through the eyes or smell through the nose. It is the processing that allows us to process these signals into something meaningful. This information is then tried to fit in with our prior knowledge, reducing cognitive dissonance. But in the process of fitting in new information, it can get twisted. Filling in what a face looks like under a mask, for example, is the brain twisting information to fit it in with what it already knows. This phenomenon happens all the time and is the science behind confirmation bias.
Human brains are extremely susceptible to confirmation bias, even going so far as to discard or ignore information that contradicts our beliefs (The science of stereotyping, 2022). The Smithsonian has a super cool interactive article about the science of bias if you are interested in learning more!
All of this to be said, our brains are amazing things, but they are not perfect. It is important to know that sometimes they take the easy path, twisting information to fit what we already know, from what someone looks like under a mask, to knowing a face-shaped rock is not a person.
References
Zhou, L.-F., & Meng, M. (2020). Do you see the “face”? Individual differences in face pareidolia. Journal of Pacific Rim Psychology, 14, e2. https://doi.org/10.1017/prp.2019.27
Kato, M., & Mugitani, R. (2015). Pareidolia in infants. PLOS ONE, 10(2), e0118539. https://doi.org/10.1371/journal.pone.0118539
Kobayashi, M. et al. (2012). Do infants recognize the Arcimboldo images as faces? Behavioral and near-infrared spectroscopic study. Journal of Experimental Child Psychology, 111(1), 22-36. https://doi.org/10.1016/j.jecp.2011.07.008.
Zhang, Q. et al. (2024). Understanding of facial features in face perception: insights from deep convolutional neural networks. Frontiers in Computational Neuroscienc, 18 https://doi.org/10.3389/fncom.2024.1209082
University of Wisconsin–Madison Department of Psychology. (2022). The science of stereotyping. https://psych.wisc.edu/news/the-science-of-stereotyping/
Smithsonian Institution. (2021). The science of bias. https://biasinsideus.si.edu/online-exhibition/the-science-of-bias
