How Does Facial Recognition Work In the Human Brain? Part 1 — Static Facial Recognition
Human’s are amazing at recognizing faces. Or even imagining faces in places where there are no faces. We are just overly sensitive to faces!
Have you ever gotten freaked out at night or while walking in the dark because you thought you saw a face where there wasn’t one? That’s thanks to several special structures in the brain, namely:
- the Occipital Face Area
- The Fusiform Face Area
- The Superior Temporal Sulcus
- The Inferior Temporal Cortex
…we can distinguish almost instantaneously whether an image contains facial data or not. We can also, using these brain structures, determine the intent of a person, to some extent what they are thinking, and whether they are someone we know and care about or whether they are a stranger.
For the purpose of Part 1 of this article, we’ll discuss static facial features. In Part 2, we’ll discuss the brain’s processing of dynamical facial features, and in Part 3 we’ll talk about the basics of face recognition in ANNs — artificial neural networks and image processors.
Note: I’m thankful to live in the modern day and age when all this information is freely available and composable, thanks to the efforts of countless others who came before me. All relevant links to primary and secondary sources have been posted on the term, and for any that sources I have missed citing, I will work to update this to include those references.
Disclaimer: None of the literature contained in this article is medical advice or to be taken as medical advice. This article is for informational purposes and potentially subject to errors which I’m constantly working to correct. Consult a physician if you have a medical issue
Sources: Many researchers have contributed to this vast and complex body of work, but a great deal of it builds on the finidings of Nancy Kanwisher and her team. Here are some of her amazing lectures on the subject of Neuroanatomy
Recognition Of The Parts of A Face: Role of the OFA (Occipital Face Area)
The process begins when light from an image enters through the retina, where it is converted into electrical impulses and then enters the occipital lobe which is in the rear of the brain. From that point on, the information is directed to image processing centers of the brain for further feature deconstruction and discrimination.
The first major destination of information for the facial processing and recognition pathway is the occipital face area. The Occipital Face Area is a region of the occipital lobe — Broadman Area 19 in brain geolocation speak — that’s responsible for recognizing features of a face and creating a sort of high level facial structure to let the person know they are looking at something that could be a face.
This ability of the OFA is part of the reason we can see faces in everything from tree trunks to grill cheese sandwiches, to the fronts and backs of cars. Incidentally this phenomenon is called Pareidolia, and is probably the inspiration for countless horror movie tropes — but I digress.
The OFA works in conjunction with the Superior Temporal Sulcus and the Fusiform Face Area to further help distinguish, and recognize actual human faces. At this point one might be asking why this doesn’t happen all in one location? Wouldn’t it be easier if the brain just processed all image data all at once instead of distributing the facial recognition function over a large network of neurological structures.
The answer is quite complex from an evolutionary perspective. The first thing to consider is the fact that evolution is an additive, not corrective mechanism of generational improvement of an organism’s species. This means that older evolutionary mechanisms won’t really be undone and a new structure be constructed when an adaptation is favorable. It rather means there will be a corrective enhancement (like for example a new nerve or new tissue structure) built around the old one. This is why vestigial organs are still present in people.
Once the OFA discriminates something as a face, the result is then forwarded to the next structures for processing. One thing to note is that the OFA to FFA and STS data transfer isn’t unidirectional. There are complex feedback mechanisms that allow FFA and STS processed data to fed back into the OFA for further facial feature isolation. These structures work in conjunction with each other, and they work very fast. Someone designing neural networks for image recognition may even be able to appreciate the computational analog to the biological cortical image recognition network that all these pieces fall under.
The Fusiform Face Area
The Fusiform Gyrus is an area nervous tissue that rests at the base of the occipital and temporal lobes — the parts of the brain that are responsible for visualization, and memory/comprehensions, respectively.
It makes sense that a neural circuit that needed to do a bit of both, was situated somewhere intersecting both regions.
While the OFA discussed previously recognizes the individual structures of the face — like things that could represent a nose, eyes and mouth — it’s the job of the FFA to stitch together all that data and actually determine and recognize a familiar face and store an unrecognized face based on interactions the person has with target face. Thus this region is of prime importance for facial perception.
The discovery of the function of the Fusiform gyrus is relatively recent. Scientists didn’t know or even really think until the late 90s that the brain had distinct structures for recognizing faces.
Advancements in FMRI (Functional Magnetic Resonance Imaging) allowed researchers to examine parts of a patient’s brain that would light up in response to certain stimuli. In this case, when people were exposed to faces, there was a very specific small area of each hemisphere of the brain that lit up like a Christmas tree. When people recognized familiar faces, the Fusiform Face Area would show the greatest activation and even more so if the patient had more personal experience with the subject they were observing, or also strangely enough when the subject was famous or well known.
This was an astounding find and researchers using their own brains, realized that this activation must happen as a result of the recognition, and in some cases the near instantaneous recall of a face.
Interesting to note that in order to differentiate between the function of the OFA and FFA, different orientation of faces were tested, as well as face-like objects vs actual faces. When a properly oriented (upright and non-distorted) face of a subject known to the patient was observed, the FFA lit up upon recognition of the subject.
When a new known face was introduced and the face was distorted or rotated, the FFA activity significantly decreased — as did recognition of the person it was attached to. In both cases the OFA activity remained the same, which lead the experimenters to the conclusion that the FFA is definitely used for facial recognition, where as the OFA was used for discrimination. Regardless of the orientation, the OFA knows we are looking at a face, while the FFA tells us which face we are looking at.
The Reactivation Hypothesis
An interesting thing to note is that there’s a distinction in FFA activity that occurs when a person is actually looking at facial data vs simply imagining a face. The FFA is decidedly more intensely activated when looking at a face in real time, as compared to through memory recall. This seems like it could make sense — especially when comparing the activation of other regions of the brain associated with working memory regions which increases in intensity of activation when trying to imagine a face, and other structures in the prefrontal cortex.
This internal activation of the recognition pathways due to imagination data is called the Re-activation hypothesis. Essentially through the process of imagination (recall of a stored memory), we reactivate the same pathways that visualization of external facial input uses to perceive an actual face.
The veracity of the reactivation hypothesis is supported by FRMI of healthy patients being asked to imagine a face as opposed to observe a face, and as well as studying cases of patients with brain damage to the FFA regions that result in prosopagnosia — face blindness. It’s almost always the case that people who have trouble distinguishing faces also have trouble imagining faces. This leads to the conclusion that imagination and perception are using the same neural structures (the FFA) and pathways to perform their tasks.
Beyond Simple Recognition: The Superior Temporal Sulcus
While at this point, the capability to even recognize faces sounds like an extraordinary feat for the brain to accomplish, it becomes even more interesting when we realize that faces not only carry structural information to distinguish the structure of a face, but that they also carry:
- Identifying information to distinguish the person attached to the face *
- Emotional signals to distinguish the person’s state of mind and the states of desire of a person (happiness, fear, anger)
- Action signals such as which direction the eyes are pointed to help discriminate what the target seems to be interested in.
- How fast the eyes are moving, or ticks of the tongue or the nose which may indicate social signals
- Biographical information which can give information about the history of the person (tattoos, whether the person is fashion sensitive, what a person’s background is, etc.)
Note that many of the above information are categorized as dynamical facial features, or ones that are motion dependent. For the next time we’ll discuss the involvement of the Superior Temporal Sulcus in distinguishing the above features, as well as neurological disorders similar to prosopagnosia that result from damage to the STS.
Hoped you enjoyed this Article. Thanks for reading!
