The National Institute of Standards and Technology (NIST) and the Department of Homeland Security (DHS) released a report on the accuracy of facial recognition algorithms in identifying individuals with face masks.
NIST said Monday the report, titled “Ongoing Face Recognition Vendor Test (FRVT) Part 6A: Face recognition accuracy with face masks using pre-COVID-19 algorithms”, found that the top 89 algorithms were able to perform with error rates of 5 to 50 percent.
As part of the FRVT, researchers tested 6M algorithms’ ability to match photos of individuals with face masks with photos of the same people without masks. The team used a variety of mask types and colors through a “one-to-one” matching procedure during the study.
According to the report, the “most accurate algorithms” produced results with 0.3 percent error rates for unmasked images and 5 percent for masked samples. The “otherwise competent” algorithms reported error rates of 20 to 50 percent.
“With respect to accuracy with face masks, we expect the technology to continue to improve,” said Mei Ngan, a computer scientist at NIST. “But the data we’ve taken so far underscores one of the ideas common to previous FRVT tests: Individual algorithms perform differently. Users should get to know the algorithm they are using thoroughly and test its performance in their own work environment.”
Ngan said that subsequent tests are scheduled for later this summer. DHS components that took part in the study include the Science and Technology Directorate, Office of Biometric Identity Management and Customs and Border Protection.