Get Avast One to help protect your personal data and sensitive accounts
- Security
- Privacy
- Performance
If you’ve ever unlocked your phone with a facial scan or had your facial image captured while going through airport security, you’ve had some experience with facial recognition. Facial recognition technology is growing quickly in both sophistication and popularity. Keep reading to learn how face recognition works and how dedicated privacy software can help safeguard your identity.
Facial recognition (or face recognition) technology is a method used to map, identity, or verify a person’s facial structure. With face recognition technology, it’s possible to create a unique numerical code, called a faceprint. These faceprints are stored in a face recognition database. If you enter a photo into the database it will find any matching faceprint it has stored.
As facial recognition technology has evolved, multiple methods have emerged to map faces and store facial data with varying levels of accuracy and efficiency. Here, we’ll explore how facial recognition works, along with how the resulting data is stored and who typically has access to it.
There are three methods of facial recognition:
Traditional facial recognition: identifying a person from a photograph.
3D facial recognition: using 3D scanners to capture a person’s likeness in real time.
Biometric facial recognition: analyzing unique facial contours, characteristics, and measurements, to determine someone’s identity.
We can split traditional facial recognition into two major categories: holistic and feature-based.
Holistic facial recognition analyzes a subject’s whole face to find identifying features that match the target.
Feature-based facial recognition separates the relevant recognition data from the face, then applies it to a template that’s compared against potential matches.
To identify a target, a traditional facial recognition system generally goes through the following steps.
Facial recognition software detects your face in an image — think of those little squares that come up when you tag Facebook photos, or the squares on your phone’s camera when you line up a picture.
Facial recognition algorithms are used to identify your unique facial biometrics and features, such as the space between your nose and mouth, the size of your eyebrows, the width of your forehead, and numerous other attributes.
These distinctive features are called nodal points, and the average human face contains roughly 80 of them. This analogue information is converted to digital code to form your faceprint.
Facial recognition software can now compare your faceprint to other faceprints in the database to find a match.
Facial recognition software matches your faceprint to others in a database.
The 3D facial recognition method involves using sensors to capture the shape of the face with more precision.
Unlike traditional facial recognition methods, the accuracy of 3D facial recognition is not affected by lighting, and scans can even be done in the dark. Another advantage of 3D facial recognition is that it can recognize a target from multiple angles, rather than just a straight-on profile.
The iPhone X (and later versions) come with Face ID technology, which relies on 3D facial recognition to identify its owner.
The 3D facial recognition process has six main steps.
Your face can be captured directly as a 3D image by facial recognition cameras (such as when you perform a face scan with your phone), or it can be captured by scanning a 2D photo.
The face recognition software now determines the position and angle of your face, as well as its size. As long as your face is oriented within 90 degrees of facing the camera, 3D facial recognition software can identify it.
Now that your image has been detected, the system measures (down to the sub-millimeter) the particular shape of your face. Once it has this very accurate measurement, a template is created.
Similar to the analysis step of traditional facial recognition, the system now takes the unique template it has created based on your facial features and converts it into code.
As you might imagine, the matching step involves searching the database to find a match for your newly converted template. If the database being searched is made up entirely of 3D images, a match can be made without any extra steps.
If the database also has 2D images, the software uses an algorithm to convert your 3D facial image into 2D to find a match.
Now, depending on the situation, the 3D facial recognition software can either verify or identify you. Verification is used to confirm your identity, matching your scan to an image that verifies who you are (like a driver’s license photo).
The software can also be used to identify a person, in which the scan is compared to all the photos in the database to find possible matches.
Skin and face biometrics is a developing genre in the field of facial recognition and can significantly increase the accuracy of face recognition technology. A skin texture analysis focuses on a section of a person’s skin, using an algorithm to make incredibly minute measurements of the lines, textures, and pores.
Biometric facial recognition analysis is so accurate it can tell the difference between identical twins.
Facial recognition data is likely in your hand or pocket right now. Many smartphones include biometric login options, including fingerprints and faceprints. That data is stored in the device itself, so your biometrics can be verified without the data being sent to an outside source.
But even though Apple and Samsung don’t (necessarily) have your faceprint sitting in a massive database, there’s a decent chance it’s out there somewhere.
Many law enforcement agencies have a face recognition database. In the United States, the FBI keeps a growing database that can match over 640 million photos. China recently made face scans mandatory for those who want a mobile data plan, and the country also keeps a facial recognition database for identifying or verifying just about every one of their 1.4 billion citizens.
There are also a number of public facial recognition databases that can be accessed for free, though the total number of images is limited compared to private or official databases.
One of the more alarming developments with face recognition databases concerns data broker companies like Clearview AI, which boasts a database of over three billion images scraped from social media, employment records, and news sites across the web.
Data brokers don't just collect images. Almost all your behavior online — from what you watch to what you buy to what sites you visit — is tracked by advertisers using cookies. That information often gets combined with data from browser fingerprinting, a deeper form of online tracking that uses hidden scripts to identify you through the unique attributes of your browser and computer.
Browser fingerprinting involves using your browser settings, default language and timezone, installed browser extensions, operating system, graphics card, and many other attributes to develop a profile that can identify you with an extremely high level of accuracy.
You can disrupt and prevent this type of data collection by using a private, secure browser that incorporates anti-fingerprinting technology. Avast Secure Browser was specially designed to prevent intrusive web trackers, confuse browser fingerprinting scripts, and block ads. For a truly private web experience, try our free secure browser today.
The accuracy of facial recognition technology has improved quite a bit over the past few years. In 2014, the top-performing algorithm had an error rate of 4.1%; in 2020, the best algorithm reported a mere 0.08% error rate. But the technology still has critical (and highly significant) shortcomings, especially related to race and gender.
Facial recognition is very accurate at identifying white men, but less so for women or people of color.
With the technology rapidly becoming ubiquitous, it’s important to ask ourselves: How accurate is facial recognition anyway? If the image analyzed is perfectly clear, the subject can be identified with up to 99.97% accuracy.
But images are rarely perfectly clear. In the real world, photos are often taken in less-than-ideal conditions (poor lighting, awkward facial profiles, and so on), and false face recognition matches can easily happen.
Two of the biggest issues in terms of accuracy are race and gender. It turns out that facial recognition is very accurate at identifying white men, but less so for women or people of color. In fact, a 2018 MIT study discovered that when a facial recognition algorithm attempts to match dark-skinned women, there’s an error rate of 34.7%! Compare that to the maximum error rate for matching white men, which sits at a miniscule 0.8%.
With the popularity of facial recognition increasing, it’s unsurprising that numerous industries see potential in the technology. There are a number of uses for facial recognition software, ranging from law enforcement to airport security to smartphones and other consumer technology. The following are just a selection of the many applications for facial recognition technology.
Facial recognition cameras in smartphonesNewer smartphones can be unlocked with face recognition software. Similar to the fingerprint scan option, using a faceprint makes it very difficult for your phone to be hacked — unless someone is able to hold your phone up to your face. Force-unlocking your phone with facial recognition was deemed illegal as a police tactic in the US in 2019.
Airport securityFacial recognition technology is used in airports around the world. Checking passengers in through facial recognition could create faster, smoother airport security procedures, privacy concerns notwithstanding.
Law enforcement agenciesAs mentioned previously, facial recognition police work is becoming increasingly popular. Law enforcement agencies use various databases for identifying and verifying persons of interest.
Identifying diseasesThe National Human Genome Research Institute has successfully used facial recognition to diagnose an ailment known as DiGeorge syndrome, making early identification of the disease more likely. In their small study, the racial recognition algorithm was able to diagnose the disease with 96.6% accuracy.
Finding petsFindingRover is a facial recognition app focused on animals. Pet owners can register and upload a picture of their pet. Then, if the pet is ever lost, the company can use facial recognition to find matches at local animal shelters.
Let’s look through a brief historical timeline of facial recognition technology.
Facial recognition takes its initial step, as slabs of obsidian are polished to form the first mirrors.
All right, all right, skipping ahead a bit…
Woodrow W. Bledsoe leads a team to determine if computers can recognize a human face. To do so, he teaches a computer to identify the features of a face and then measure the features and compare them to other faces. The experiment is not a success, but the idea shows promise and increases interest in face recognition software.
Bledsoe’s work is built upon by Goldstein, Harmon, and Lesk, who develop 21 specific face markers for computers to use in recognition. Unfortunately, like Bledsoe before them, they’re limited by the technology of the time, which requires a good deal of manual computing.
Eigenfaces! Sorry, that’s just a fun word to yell. In 1988, Surovich and Kirby develop the idea of “eigenfaces,” which are essentially standardized faces built from studying numerous other faces. This is the first holistic (whole-faced) approach to facial recognition.
In 1991, Turk and Pentland expand on this work, discovering a way to detect faces in images. Though these experiments are also limited by the available technology, they show success. Today, eigenfaces are still considered a baseline comparison in some facial recognition methodologies.
Several advances in facial recognition occur at the turn of the last century. In 1993, the Defense Advanced Research Projects Agency (DARPA) and the National Institute of Standards and Technology (NIST) launch the FERET program, with the goal of building a facial recognition database.
Later, in the early 2000s, the National Institute of Standards and Technology introduces a series of evaluations called the Face Recognition Vendor Test. These independent assessments of facial recognition databases are still in use today.
Here’s where things really start taking off. Facebook launches facial recognition software that tags users when they (or someone else) upload a photo.
Apple announces that the iPhone X will include Face ID, their own facial recognition software that allows users to unlock their phones simply by looking at them. 2017 also witnesses the birth of Clearview AI, mentioned above, and its database of over three billion photos scraped from the internet.
There are certainly incredible things that can be accomplished with facial recognition, but there are many understandable concerns that facial scans can be misused or lead to false matches. Fortunately, there are ways to protect your data.
During the COVID-19 pandemic, facial recognition technology has been used to determine if employees are following face-mask protocols. The technology was covertly used in hotels and restaurants across Europe and the USA, prompting privacy concerns from critics.
In China, surveillance technologies like facial recognition have been used to find and convict religious minorities. The Chinese government is also selling their surveillance tools to authoritarian governments around the world, including Venezuela and Zimbabwe.
Governments can use facial recognition technology to surveil their citizens.
IBM has said that it will no longer develop or offer facial recognition technology. Google won’t offer facial recognition until it can develop policies to prevent misuse. Even Amazon has placed a one-year ban on police using their facial recognition software, due to concerns of accuracy and racial bias.
If facial scanning freaks you out a bit, there are a few simple things you can do to reduce your chances of being added to a database.
Opt out of facial recognition on Facebook. You can change your Facebook settings so that you’ll no longer be recognized by Facebook’s face identification software. When you select this option, Facebook also deletes the face template they have for you.
Stop turning yourself into an alien or dog. Though Snapchat has said their face filter technology doesn’t go as far as facial recognition algorithms do, an app like this could do some questionable things with their millions of face scans if they so choose. Face App raised some eyebrows when it went viral last year for a filter that aged the user. Concerningly, their terms of use give the company the right to use uploaded photos however they see fit.
Stay anonymous online with a VPN (virtual private network). VPNs encrypt the data surrounding your online activities and hide your IP address, which helps to conceal your identity from malicious actors, and even from your ISP (internet service provider). By doing so, you can prevent web tracking and limit the amount of information that can be linked to your image.
But even with a strong and reliable VPN, you should still be wary when adding images of yourself online. As is often said online, “the internet is forever.”
Data brokers, companies, and others gather data about everybody online, including you. And they sell this data to marketers, advertisers, and other players interested in using aggregated or personalized insights. As if that’s not bad enough, data brokers’ databases can get exposed, with the data winding up for sale on the dark web.
Avast BreachGuard gives you more control over your online data by contacting known data brokers to demand that they remove your data from their registries. At the same time, it monitors marketplaces on the dark web and alerts you if any of your personal information has leaked — empowering you to face the problem of online privacy head-on.
