Recent research published in journals brain research This provides evidence that people can recognize facial expressions faster and more accurately when they see the entire face rather than just the eyes. This study suggests that while the eyes contain important social cues, the human brain relies heavily on the surrounding facial context to efficiently process and evaluate emotions.
In everyday life, people rely heavily on nonverbal communication to understand the emotional states of those around them. In these everyday social interactions, the eyes tend to be the main focus. Visual changes such as furrowed eyebrows, wide eyelids, and small wrinkles at the corners of the eyes convey rich emotional information.
When people look at faces, compositional processing usually occurs. This concept refers to the brain’s ability to perceive the entire face at once and understand how all the different features relate to each other in a unified layout. When visual information is limited, the brain has to rely on feature processing, which analyzes individual parts of the body, such as the eyes and nose, individually.
Katherine A. Billetdeau, a doctoral student in developmental psychology and social behavioral neuroscience at Penn State, began this research while attending the College of Worcester. She wanted to understand exactly how much weight the human brain gives to the eyes compared to other parts of the face.
“This idea came from my experience wearing a mask for all in-person interactions during the COVID-19 pandemic,” Billetdeau said. “I noticed that when the lower half of someone’s face is covered, it’s much harder to read certain facial expressions, and communication feels less clear.”
“We also started wondering how people who don’t normally make eye contact interact with masked faces. That led us to think about what kind of facial information we actually rely on to recognize emotions, and to investigate that more systematically.”
Billetdeau said the project began as an independent research paper during his undergraduate years. “Experiencing all stages of the research process first-hand, from design to data collection and analysis, gave me a deeper understanding of what scientific investigation is actually like,” she said.
“Not many undergraduates get the opportunity to see their projects through to publication, and I’m lucky to be able to share my work with a wider audience,” Billetdeau said. “More than anything, it confirmed that research is what I want to do, and it was a big part of my motivation to get my PhD and continue this research.”
To investigate these visual mechanisms, the researchers designed an experiment to test how facial expression recognition changes when certain visual information is removed. They wanted to see what happens in the brain when the eyes are slightly obscured compared to when the rest of the face is completely obscured. The main goal was to track both behavioral accuracy and the underlying brain waves that occur during these specific visual tasks.
The authors recruited 40 undergraduate students for the experiment. The sample included 10 men, 29 women, and one non-binary individual. Participants had an average age of 19.4 years and were compensated with course credit for their time participating.
Each participant sat in front of a computer monitor and viewed a series of 480 facial photographs. These photos featured 30 different models expressing four specific emotions: anger, fear, happiness, and sadness. The researchers precisely aligned the images so that the eyes and nose appeared in the same position on the screen, minimizing sudden eye movements.
Scientists manipulated digital images to create four different viewing conditions. In the first state, the entire face was visible and not modified in any way. In the second condition, the entire face is visible, but the eyes are scaled down using a white rectangle with 20% transparency to blur small details.
In the third condition, the researchers showed only the intact eye, while the rest of the face was hidden by a plain white mask with 0% transparency. Finally, in the fourth condition, only eyes are visible, but those isolated eyes are also hidden by a translucent blur, resulting in a sliding scale of visual degradation.
Participants were instructed to identify the emotion displayed on the screen as quickly as possible by pressing specific buttons on a computer keyboard. Prior to the main experiment, participants completed practice to learn which finger corresponds to which emotion button. In the actual test, each face remained on the screen until the participant made a selection.
As participants completed the recognition task, researchers recorded their brain activity using electroencephalograms. This specialized machine uses non-invasive sensors placed on the scalp to measure tiny electrical signals generated by the brain. The scientists focused specifically on event-related potentials, which are distinct spikes or dips in the brain’s electrical activity that occur in response to specific visual events.
They measured an early brain wave known as the N170. This particular electrical signal occurs about 170 milliseconds after a person looks at a face. The size and speed of this wave reflects the brain’s early structural processing of visual images.
The researchers also measured two late brain waves known as P300 and late positive potentials. These electrical signals typically occur between 250 and 800 milliseconds after viewing the image. These are indicative of higher-order brain processes, such as paying close attention, assessing the meaning of images, and processing emotional significance.
Behavioral results showed that participants were most accurate and had fastest reaction times when the entire face was visible. When the facial context was removed, participants made significantly more errors and took longer to respond. Similarly, reducing eye detail will result in slower and less accurate reactions compared to when the eyes are fully visible.
“The key takeaway is that both the eye area and the surrounding face are important to how we process facial expressions. But the eyes play a particularly important role when we’re experiencing an internal reaction to the emotion we’re seeing,” says Billetdeau. “We also found that different types of expressions are not all processed in the same way, so this gives us more nuance than a simple ‘eyes are most important’ conclusion.”
Recorded EEG corroborated these behavioral observations. When participants looked at the whole face, early structural brain waves were smaller and occurred faster. This provides evidence that the brain requires less effort to process emotions when all facial features are present together.
When participants looked only at the eyes and not at the faces around them, these early brain waves became larger and slower. This suggests that the brain has a much harder time structuring and understanding faces when their global organization is missing. Interestingly, blurring the eyes only disrupted this early processing step when the rest of the face was obscured.
“The most surprising finding was that the reduction of the eye area affected different stages of brain processing, depending on whether the rest of the face was visible or not,” Billet-deau told PsyPost. “In other words, the brain doesn’t process emotional information from the eyes in isolation; the surrounding facial context actually changes how that information is processed. This kind of interaction between facial features wasn’t something we completely expected.”
Subsequent brain waves told a slightly different story about how the brain assesses the emotional meaning of faces. These later electrical signals were larger when the eyes were blurred, but only when the rest of the face was visible. This suggests that once the brain completes its initial structural scan, it pays particular attention to the details of the eyes to fully assess emotion.
The authors also found that different emotions depend on different facial features. Anger was easily recognized whether the entire face was shown or just the eyes. This suggests that expressions of anger are highly concentrated in the eye region and do not require much surrounding context.
Fearful faces were found to be the most difficult to recognize when the background of the surrounding faces was obscured. When participants looked only at the eyes of fearful faces, their brain activity increased significantly, indicating significantly higher levels of cognitive effort. This tends to suggest that a wide open mouth or other lower facial features may be necessary for fear to be easily recognized.
Happiness was identified much faster and more accurately when the entire face was visible. Because happy facial expressions usually have a prominent smile, the mouth appears to be the strongest signal for this particular emotion. Sadness is relatively easily processed compared to fear, providing evidence that sad facial expressions are strongly conveyed through the eyes.
Although this study provides detailed insights into social cognition, there are some limitations that should be considered. The techniques used to hide the eyes changed the contrast and transparency of the image, rather than precisely filtering out specific visual frequencies. This makes it difficult to isolate precisely the visual characteristics responsible for the observed changes in brain activity.
“One technical caveat worth noting is that we cannot say with certainty why eye reduction had such an effect,” Billet-deau said. “This operation changed contrast and opacity, which is similar to but not the same as reducing spatial frequency.”
In visual science, spatial frequency refers to the level of sharp detail present in an image. “Spatial frequency and contrast/opacity are distinct visual properties, and the brain may process them differently,” Billet-Deau added. “To study the exact mechanism, we need to systematically compare different methods of concealing the eye area.”
The study also did not measure the emotional intensity or arousal level of the photos. Some faces may have evoked stronger overall arousal or vigilance than others, which may have influenced the EEG recordings. Future research may benefit from scaling emotional intensity to see how arousal interacts with face recognition.
Another limitation is related to the experimental setup, which required participants to focus explicitly on emotion identification. These findings may not apply to everyday situations where people process facial expressions naturally without direct instruction. Future research could investigate how the brain responds to these facial cues when a person is distracted or focused on another task.
Billettedeau hopes to extend these findings using functional magnetic resonance imaging (fMRI), which tracks blood flow and reveals active areas in the brain. “One of the key next steps is to use fMRI, which will tell us where in the brain this processing is taking place,” Billet-deau said. “This will complement the timing that current ERP practices reveal.”
“We are also interested in exploring how these processes work in clinical populations that have difficulty with social communication, such as people with autism,” Billet-Deau said. “Ultimately, we hope this study will help inform intervention strategies to improve social communication for everyone, not just those with a formal diagnosis.”
The study, “Are the eyes windows to the soul? The importance of the eyes in facial expression recognition,” was authored by Kathryn A. Billettedeau and Grit Hersman.
