Multidisciplinary study provides scientific underpinnings for accuracy offorensic facial identification.
Experts at recognizing faces often play a crucial rolein criminal cases. A photo from a security camera canmean prison or freedom for a defendant—and testimony from highly trained forensic face examiners informs the jury whether that image actually depictsthe accused. Just how good are facial recognitionexperts? Would artificial intelligence help?
A study appearing this week in the Proceedings of theNational Academy of Sciences has brought answers.In work that combines forensic science with psychology and computer vision research, a team of scientists from the National Institute of Standards andTechnology (NIST) and three universities has testedthe accuracy of professional face identifiers, providingat least one revelation that surprised even the researchers: Trained human beings perform best witha computer as a partner, not another person
. “This is the first study to measure face identificationaccuracy for professional forensic facial examiners,working under circumstances that apply in real-world casework,” said NIST electronic engineer P.Jonathon Phillips. “Our deeper goal was to find better ways to increase the accuracy of forensic facialcomparisons.” The team’s effort began in response toa 2009 report by the National Research Council,Strengthening Forensic Science in the United States:A Path Forward (link is external), which underscoredthe need to measure the accuracy of forensic examiner decisions.
The NIST study is the most comprehensive examination to date of face identification performanceacross a large, varied group of people. The study alsoexamines the best technology as well, comparing theaccuracy of state-of-the-art face recognition algorithms to human experts.
Their result from this classic confrontation of humanversus machine? Neither gets the best results alone.Maximum accuracy was achieved with a collaborationbetween the two.
“Societies rely on the expertise and training of professional forensic facial examiners, because theirjudgments are thought to be best,” said co-authorAlice O’Toole, a professor of cognitive science at theUniversity of Texas at Dallas. “However, we learnedthat to get the most highly accurate face identification, we should combine the strengths of humans andmachines.”
The results arrive at a timely moment in the development of facial recognition technology, which hasbeen advancing for decades, but has only very recently attained competence approaching that of top-performing humans.
“If we had done this study three years ago, the bestcomputer algorithm’s performance would have beencomparable to an average untrained student,”Phillips said. “Nowadays, state-of-the-art algorithmsperform as well as a highly trained professional.”
The study itself involved a total of 184 participants, alarge number for an experiment of this type. Eighty-seven were trained professional facial examiners,while 13 were “super recognizers,” a term implyingexceptional natural ability. The remaining 84—thecontrol groups—included 53 fingerprint examinersand 31 undergraduate students, none of whom hadtraining in facial comparisons.
For the test, the participants received 20 pairs of faceimages and rated the likelihood of each pair beingthe same person on a seven-point scale. The researchteam intentionally selected extremely challengingpairs, using images taken with limited control of illumination, expression and appearance. They thentested four of the latest computerized facial recognition algorithms, all developed between 2015 and2017, using the same image pairs.
Three of the algorithms were developed by RamaChellappa, a professor of electrical and computer engineering at the University of Maryland, and histeam, who contributed to the study. The algorithmswere trained to work in general face recognition situations and were applied without modification to theimage sets.
One of the findings was unsurprising but significantto the justice system: The trained professionals didsignificantly better than the untrained controlgroups. This result established the superior ability ofthe trained examiners, thus providing for the firsttime a scientific basis for their testimony in court.
The algorithms also acquitted themselves well, asmight be expected from the steady improvement inalgorithm performance over the past few years.
What raised the team’s collective eyebrows regardedthe performance of multiple examiners. The teamdiscovered that combining the opinions of multiple forensic face examiners did not bring the mostaccurate results.
“Our data show that the best results come from asingle facial examiner working with a single top-performing algorithm,” Phillips said. “While combiningtwo human examiners does improve accuracy, it’s notas good as combining one examiner and the bestalgorithm.”
Combining examiners and AI is not currently usedin real-world forensic casework. While this study didnot explicitly test this fusion of examiners and AI insuch an operational forensic environment, resultsprovide an roadmap for improving the accuracy offace identification in future systems.
While the three-year project has revealed thathumans and algorithms use different approaches tocompare faces, it poses a tantalizing question to otherscientists: Just what is the underlying distinction between the human and the algorithmic approach?
“If combining decisions from two sources increasesaccuracy, then this method demonstrates the existence of different strategies,” Phillips said. “But it doesnot explain how the strategies are different.”
The research team also included psychologist David White from Australia’s University of New South Wales.
Paper: P.J. Phillips, A.N. Yates, Y. Hu, C.A. Hahn, E. Noyes, K.Jackson, J.G. Cavazos, G. Jeckeln, R. Ranjan, S. Sankaranarayanan, J.-C. Chen, C.D. Castillo, R. Chellappa, D. White andA.J. O’Toole. Face Recognition Accuracy of Forensic Examiners, Superrecognizers, and Algorithms. Proceedings of the NationalAcademy of Sciences, Published online May 28, 2018. DOI:10.1073/pnas.1721355115 (link is external)
