Use of foveal descriptors help improve facial recognition

Credit: Isabelle Vincent/ Credit: Isabelle Vincent/

Deva Ramanan, an associate professor of robotics at Carnegie Mellon, and Peiyun Hu, a Ph.D. robotics student, recently developed an improved method for detecting multiple faces in a crowd. The feat of distinguishing small objects is one challenge in computer vision, which is why Ramanan and Hu’s achievement is so crucial.

The method they developed uses “foveal descriptors.” Just as the human brain cause the eye to focus on an object and blur the peripheral vision, so does this method, which allows a computer to process specific pixels in search of facial features.

The team also developed specific detectors for different sized faces. For example, a detector was made to find faces a few pixels across, while another focused on faces with much higher resolution.

Improved computer vision has many benefits. Self-driving cars need exceptional vision to avoid accidents, and at high speeds the images they process may be slightly blurred. Agencies like the Federal Bureau of Investigation could use improved face detection to better identify suspects, even if the only images available are from a grainy closed-circuit television feed. Ramanan and Hu’s developments may offer a solution.

Computer vision is arguably one of the fastest growing fields in computer science. Today, phones like the Samsung S8 use facial and iris detection to restrict access. A use of face detection many of us are familiar with is in Snapchat filters, which find and distort faces. Another well known use is in tagging on Facebook. Facebook runs a program that detects when a face in an image has not been tagged. It is also capable of pattern-matching like faces to generate tag suggestions.

The science behind face detection is even more fascinating. Analysis of millions of images has enabled computer scientists to determine the average features of the human face, such as dark regions around the nose bridge and under the eyebrows. Computers detect these features in pixel color values to detect a face. Unfortunately, this method makes it difficult to detect a face if it's not facing the camera head-on.

Ramanan compares the feat of face detection in a crow to “spotting a toothpick in someone’s hand." Ramanan and Hu’s complete publication, titled “Finding Tiny Faces,” is available online.