Facial-recognition, social media sites create potential security risk

Amritha Parthasarathy Aug 20, 2011

In this generation, many people — from students to professionals to children — are addicted to social media sites such as Facebook, Twitter, LinkedIn, or MySpace. These sites offer us an opportunity to connect with people we know and, sometimes, people we don’t know. On a daily basis, we share huge amounts of information through these connections, including personal photographs, addresses, and phone numbers. All this information makes social media sites a magnet for hackers and government agencies.

But just how accessible is this information to outsiders?

Facial recognition software can be a successful tool for extracting information from a picture by associating it with a specific name, especially when combined with other software to obtain personal information linked to the name. According to the Los Angeles Times, “Facial recognition software is growing and is being used and further developed by Facebook, Google, Apple, and the U.S. government.” Indeed, many popular programs such as Google’s Picasa, Facebook, and Apple’s iPhoto use facial recognition software.

This advent of this technology creates a sizeable risk in the accessibility of personal information. Alessandro Acquisti, an associate professor of information technology and public policy in the Heinz College and a Carnegie Mellon CyLab researcher, has conducted several experiments to examine the potential of facial recognition software to identify faces and match them with personal information.

In the first experiment, Acquisti’s team identified individuals on an online dating site where members use pseudonyms to protect their privacy. The research team took profile pictures from the dating site and cross-referenced the pictures with Facebook profile images accessible through a search engine using facial recognition software. As a result, they “ended up re-identifying a statistically significant proportion of members of the dating site,” according to Acquisti’s website, thus showing the viability of online-to-online re-identification.

In the second experiment, the group implemented offline-to-online re-identification by identifying students walking around campus based on their photos on Facebook. In this case, the team re-identified students from images taken through a webcam on the Carnegie Mellon campus before comparing them to profile images from Facebook. According to Acquisti’s website, they were able to identify around one-third of the subjects in the experiment.

In the third experiment, Acquisti’s team worked on what the team’s website calls “augmented reality,” or “the merging of online and offline data that new technologies make possible.” This means that if a person’s face can be linked to a name through facial recognition software that is in turn connected to social networking sites, it is possible to identify more sensitive information about the person.

To prove this, the team predicted interests and social security numbers (SSNs) of some participants from the second experiment by combining facial recognition and previously developed algorithms to predict SSNs from public data. This blend of information from pictures, facial recognition, and public data warehouses is the augmented reality the team has been studying.

Furthermore, to demonstrate the real-time processing of personal information based on faces, Acquisti’s team created a smartphone application that collects online and offline personal data. The information found is then displayed over the target person’s face on the smartphone’s screen. This application can recognize faces and withdraw information about the individual from sites where similar pictures have been posted.

With such software and information scattered across the Internet on social networking sites, we may be moving toward a future in which everyone knows everything about everyone.

At the Black Hat computer security conference in Las Vegas, Acquisti posited, “In a few years, facial visual searches may become as common as today’s text-based searches.” It is hard to imagine a future in which just a small click of a camera would result in a detailed history of a person.
In Acquisti’s words, “A person’s face is the veritable link between her offline and online identities. When we share photographs of ourselves online, it becomes possible for others to link our face to our names in situations where we would normally expect anonymity.”