When the task is to assign an image of a person wearing a digitally added face mask another photograph of a person without one, the most accurate facial recognition algorithms reportedly failed to achieve a correct match between 5% and 50% of the time. Generally speaking, most of the algorithms tested had a failure rate between 20% and 50%, informed CNN Business Mei Ngan, a computer scientist at NIST and the author of the report.
Identification problems make sense because face recognition systems usually work by comparing measurements between different facial features in one image with those in another. Blocking a part of a face means that there is less information available that the software will use to create a match.
The researchers created nine different ones for their report the shapes of the black and light blue masks that are responsible for how the shapes of the masks differ in the real world and that used them to cover part of a person̵7;s face in a photograph. They then compared a digitally masked photo of each person with another unmasked photo of the same person. They also tested the algorithms on both series of photos without virtual masks.
In total, they tested 89 algorithms on more than 6 million photos with a million different people. The photos came from two sources: applications for immigration benefits in the United States, which were used as images without a mask, and photographs of passengers who crossed the border to enter the United States, who were provided with a digital mask.
NIST found that the best of these algorithms – which were submitted to the lab before mid-March – failed only 0.3% of the time when tested on the same sets of photos without digital masks. However, with digital masks enabled, the error rate between these algorithms increased to 5%.
An obvious shortcoming of the report is that NIST did not test the algorithms on images of people actually wearing masks – Ngan said digital mask approximations were used due to time and resource constraints. On the positive side, it has allowed researchers to quickly get an idea of the effect of masks on algorithms, but real masks fit different people in different ways. It is still not known how textures or patterns can affect the accuracy of face recognition software.
“We want to look at that,” Ngan said.
This is the focus for Marios Savvides, a professor at Carnegie Mellon University who is studying biometric identification. He said that a person who has a mask can be essentially invisible in the face recognition system because he does not recognize the face at all in the first place. He thinks that the area of the face that includes the eyes and eyebrows changes the least over time, making it a good part of the face as we try to identify the person whose mouth and nose are hidden.
The NIST report is the first of several the lab plans to release on how facial recognition algorithms identify masked faces. In the fall, Ngan said, NIST expects to release a report on the accuracy of algorithms that have been specially created to find people in masks.