The print on the bag: how investigations go wrong

On 11 March 2004, bombs exploded on commuter trains in Madrid, killing about 200 people and injuring more than 1,400. Spanish National Police found fingerprints on a bag of detonators connected with the attacks and sent images of them through INTERPOL. In Washington, the FBI Laboratory ran one of them, latent print 17, through its automated fingerprint system, which searched the prints of more than 47 million people (Office of the Inspector General, 2006).

On 19 March the FBI Laboratory identified the print as belonging to Brandon Mayfield, a lawyer in Oregon. A supervisory examiner made the identification. Two more examiners agreed. The FBI put Mayfield under 24-hour surveillance. It learned that he was Muslim, and that he had represented a man convicted of a terrorism offence in a child custody dispute (Office of the Inspector General, 2006).

On 13 April the Spanish police laboratory reported that its own comparison was negativo: the print was not Mayfield's. The FBI sent an examiner to Madrid to explain its identification. On 6 May the FBI arrested Mayfield as a material witness. In court that day he said the print was not his and that he had no idea how it could be. He was held in a county detention centre. On 17 May the court appointed an independent fingerprint expert, who on 19 May agreed with the FBI (Office of the Inspector General, 2006).

That same day, the Spanish police told the FBI they had identified the print as belonging to someone else: an Algerian national, Ouhnane Daoud. Mayfield was released to home detention on 20 May. After examining Daoud's prints, the FBI Laboratory withdrew its identification of Mayfield on 24 May (Office of the Inspector General, 2006).

Four qualified examiners had looked at the same print and seen Mayfield. The US Department of Justice's Office of the Inspector General spent the next year and a half working out how.

What the Inspector General found

The report did not find a conspiracy, and it did not find that the examiners lied. It found a sequence of ordinary errors, each making the next one easier (Office of the Inspector General, 2006):

• An unusual similarity. Mayfield's print really did resemble latent print 17 closely, in as many as ten points. The automated search had done its job of finding a close candidate. That closeness was the trap.
• Circular reasoning. Having seen the similarity, examiners began to interpret murky details in the latent print by reasoning backwards from Mayfield's known prints. Features the latent did not clearly show were "found" because the known prints suggested them. The report shows the original examiner's own coding of the print changing after he saw Mayfield's.
• Tiny details. The examiners leaned on extremely fine ridge detail of the kind that is easy to over-read in a poor image.
• Differences explained away. Where the prints did not match, the differences were given explanations rather than treated as evidence against the match.
• No real second look. After the Spanish negativo report, the identification was defended rather than seriously re-examined.

The report also considered Mayfield's religion. It concluded that it was not the reason for the original identification, but that it may have contributed to the examiners' failure to revisit their conclusion once doubts were raised (Office of the Inspector General, 2006).

Same expert, same print, different answer

Itiel Dror, David Charlton and Péron used the Mayfield case to run an experiment that is hard to forget. They took five experienced latent print examiners and, without telling them, showed each a pair of prints that the same examiner had earlier identified as a match in a real case. This time the prints were presented as the notorious Mayfield prints, an example of a mistaken match (Dror et al., 2006).

Four of the five no longer reported a match. The prints had not changed. The examiners had not changed. What changed was what they had been told before they looked (Dror et al., 2006).

Saul Kassin, Dror and Jeff Kukucka later gathered the research on what they named the forensic confirmation bias. Context, they found, shapes forensic judgement across domains: knowing that a suspect has confessed, or what the detectives believe, can tilt how an examiner reads ambiguous evidence, and how lay witnesses perceive and remember (Kassin et al., 2013). In 2009 the US National Research Council had already reported that much forensic pattern-matching evidence rested on a weaker scientific foundation than courts assumed (National Research Council, 2009).

Prime suspect

Bias in the laboratory is a special case of something broader. Barbara O'Brien gave participants a mock police file and asked some of them to name a suspect early, before they had read everything. Those who committed to a suspect early went on to favour evidence that fitted that person and to discount evidence that did not. A simple counter-measure helped: asking participants to argue why their suspect might be innocent reduced the effect (O'Brien, 2009).

Keith Findley and Michael Scott call the whole pattern tunnel vision. In their account it is not the work of corrupt investigators. It grows from normal features of human thinking, confirmation bias, hindsight bias, the pull of a story once it has a shape, and is then hardened by institutions: pressure to solve a case, the cost of admitting error, and roles that reward building a case rather than testing it (Findley & Scott, 2006).

Once an investigation narrows, its products can look like independent confirmation when they are not. A confession can come out of an interrogation built around the suspect police already believe guilty. The research on false confessions identifies the risk factors: youth, intellectual disability, mental illness, long interrogations, the presentation of false evidence, and minimisation of the consequences (Kassin et al., 2010). A forensic examiner who knows about that confession then reads ambiguous evidence in its light (Kassin et al., 2013). Two pieces of evidence that seem to support each other may both trace back to a single early belief.

The fixes are procedural

The research is consistent on one point: the answer is not to tell investigators to try harder to be objective. The Mayfield examiners were experienced and conscientious. The fixes are about what people know, and when (Kassin et al., 2013):

• Examine the crime-scene evidence, and write down what it shows, before seeing the suspect's material.
• Keep information the examiner does not need, such as confessions, a suspect's background or detectives' theories, out of the laboratory.
• Verify blind: a second examiner who does not know the first examiner's conclusion.
• Record every interrogation, in full, so that how a confession was produced can be examined later (Kassin et al., 2010).
• Build in a step where someone's job is to argue the case is wrong (O'Brien, 2009).

These are cheap. They ask people to accept a less dramatic picture of themselves: as instruments that can be miscalibrated by what they were told first. Without them, a justice system can produce guilt built at scale from parts that each looked sound.

Brandon Mayfield's print was close to the real one. The real one was in Madrid all along. Between them stood several careful people who had already seen what they expected to see.

Short answers

What is tunnel vision in a criminal investigation?

A focus on one suspect that leads investigators to look for, interpret and remember evidence in a way that supports that suspect, and to discount what points elsewhere. Findley and Scott (2006) describe it as a product of ordinary cognitive biases and institutional pressures, not of bad faith.

What went wrong in the Brandon Mayfield case?

The US Department of Justice Inspector General found that FBI examiners, having seen an unusual similarity, read features of Mayfield's known prints back into the unclear latent print (circular reasoning), relied on tiny details, explained away differences, and did not seriously re-examine the match after Spanish police reported a negative result (Office of the Inspector General, 2006).

How can forensic bias be reduced?

By controlling what examiners know and when: examine the crime-scene evidence and record conclusions before seeing the suspect's material, keep irrelevant case information away from the laboratory, and use blind verification (Kassin et al., 2013).

References

1. Dror, I. E., Charlton, D., & Péron, A. E. (2006). Contextual information renders experts vulnerable to making erroneous identifications. Forensic Science International, 156(1), 74–78. doi.org/10.1016/j.forsciint.2005.10.017
2. Findley, K. A., & Scott, M. S. (2006). The multiple dimensions of tunnel vision in criminal cases. Wisconsin Law Review, 2006(2), 291–397.
3. Kassin, S. M., Drizin, S. A., Grisso, T., Gudjonsson, G. H., Leo, R. A., & Redlich, A. D. (2010). Police-induced confessions: Risk factors and recommendations. Law and Human Behavior, 34(1), 3–38. doi.org/10.1007/s10979-009-9188-6
4. Kassin, S. M., Dror, I. E., & Kukucka, J. (2013). The forensic confirmation bias: Problems, perspectives, and proposed solutions. Journal of Applied Research in Memory and Cognition, 2(1), 42–52. doi.org/10.1016/j.jarmac.2013.01.001
5. National Research Council. (2009). Strengthening forensic science in the United States: A path forward. National Academies Press. doi.org/10.17226/12589
6. O'Brien, B. (2009). Prime suspect: An examination of factors that aggravate and counteract confirmation bias in criminal investigations. Psychology, Public Policy, and Law, 15(4), 315–334. doi.org/10.1037/a0017881
7. Office of the Inspector General. (2006). A review of the FBI's handling of the Brandon Mayfield case. U.S. Department of Justice. oig.justice.gov/sites/default/files/legacy/special/s0601/final.pdf