CSI Myths: The Shaky Science Behind Forensics

Even fingerprint evidence is often shaky. Forensic science was not developed by scientists. It was mostly created by cops, who were guided by little more than common sense. And as hundreds of criminal cases begin to unravel, many established forensic practices are coming under fire.

On Jan. 11, 1992, the jury in the murder trial of Roy Brown heard from a dentist named Edward Mofson. To establish his credentials, Dr. Mofson testified that he was certified in forensic odontology, belonged to six related professional organizations and did forensic consulting throughout New York state. He then explained that several months earlier he was called to the morgue in Cayuga County, New York, to analyze the body of 49-year-old Sabina Kulakowski.

Kulakowski’s corpse was found by a volunteer firefighter on a dirt road some 300 yards from the farmhouse where she lived, which had burned to the ground in the night. She was severely beaten and stabbed, and there were multiple bite marks on her body. Brown was a natural suspect in the grisly murder. The week before the crime, the hard-drinking 31-year-old had been released from jail on charges of threatening to “wipe everybody out” at the social services office where Kulakowski worked; the agency had put his daughter into foster care. In addition to the motive, the district attorney at trial produced other circumstantial evidence, including testimony from Brown’s two ex-wives that he had bitten them. But Mofson, now deceased, was the centerpiece of the prosecution.

Mofson testified that seven bite marks found on Kulakowski were “entirely consistent” with dental impressions taken from Brown. It was the only physical evidence tying Brown to the crime. Although a defense expert disputed Mofson’s findings, the jury convicted Brown of second-degree murder. He was sentenced to 25 years to life in prison.

As the years ticked by, few listened as Brown proclaimed his innocence from his cell in the Elmira Correctional Facility. Then Brown got an unusual lucky break. His stepfather’s house burned down, taking with it all of his records from the trial. To replace his documents, Brown submitted an open records request to the county. The sheriff who processed Brown’s request mistakenly sent him the entire investigative file. It revealed another suspect: Barry Bench, the firefighter who discovered Kulakowski’s body. Bench’s brother had dated Kulakowski up until two months before the murder and Bench was reportedly upset that she continued to live in the family farmhouse. On the day before Christmas in 2003, Brown sent a letter to Bench letting him know he was seeking DNA testing. “Juries can make mistakes,” he wrote. But, “DNA is God’s creation, and God makes no mistakes.” Soon after receiving the message, Bench committed suicide by jumping in front of an Amtrak train. DNA tests confirmed that Bench was guilty of Kulakowski’s murder, and Brown was set free.

The faulty identification that sent Brown to prison for 15 years may seem like a rare glitch in the U.S. criminal justice system. It wasn’t. As DNA testing has made it possible to re-examine biological evidence from past trials, more than 200 people have had their convictions overturned. In approximately 50 percent of those cases, bad forensic analysis contributed to their imprisonment.

On television and in the movies, forensic examiners unravel difficult cases with a combination of scientific acumen, cutting-edge technology and dogged persistence. The gee-whiz wonder of it all has spawned its own media-age legal phenomenon known as the “CSI effect.” Jurors routinely afford confident scientific experts an almost mythic infallibility because they evoke the bold characters from crime dramas. The real world of forensic science, however, is far different. America’s forensic labs are overburdened, understaffed and under intense pressure from prosecutors to produce results. According to a 2005 study by the Department of Justice, the average lab has a backlog of 401 requests for services. Plus, several state and city forensic departments have been racked by scandals involving mishandled evidence and outright fraud.

But criminal forensics has a deeper problem of basic validity. Bite marks, blood-splatter patterns, ballistics, and hair, fiber and handwriting analysis sound compelling in the courtroom, but much of the “science” behind forensic science rests on surprisingly shaky foundations. Many well-established forms of evidence are the product of highly subjective analysis by people with minimal credentials—according to the American Society of Crime Laboratory Directors, no advanced degree is required for a career in forensics. And even the most experienced and respected professionals can come to inaccurate conclusions, because the body of research behind the majority of the forensic sciences is incomplete, and the established methodologies are often inexact. “There is no scientific foundation for it,” says Arizona State University law professor Michael Saks. “As you begin to unpack it you find it’s a lot of loosey-goosey stuff.”

Not surprisingly, a movement to reform the way forensics is done in the U.S. is gaining momentum. The call for change has been fueled by some embarrassing failures, even at the highest levels of law enforcement. After the 2004 train bombings in Madrid, Spain, the FBI arrested Oregon lawyer Brandon Mayfield and kept him in jail for two weeks. His incarceration was based on a purported fingerprint match to a print found on a bag of detonators discovered near the scene of the crime. As a later investigation by the Justice Department revealed, the FBI’s fingerprint-analysis software never actually matched Mayfield to the suspect fingerprint, but produced him as an “unusually close nonmatch.” Lacking any statistical context for how rare such similarities are, investigators quickly convinced themselves that Mayfield was the prime suspect.

The next year, 2005, Congress commissioned the National Academy of Sciences (NAS) to examine the state of forensics in U.S. law enforcement. The result was a blistering report that came out this February, noting “serious deficiencies” in the nation’s forensic science system and advocating extensive reforms. It specifically noted that apart from DNA, there is not a single forensic discipline that has been proven “with a high degree of certainty” to be able to match a piece of evidence to a suspect. The obvious implication is the sobering possibility that more Roy Browns are currently locked up based on shoddy science. Then there’s the flip side: A lot of bad guys who should be in prison still roam free. A study by the Innocence Project of the prisoners exonerated by DNA found that the real perpetrators were identified in 103 cases—roughly half. In all but one, the perpetrator committed at least one serious crime after the innocent person was jailed....

The FBI’s errors in the Madrid bombing case were particularly surprising because they called into question one of the gold standards of evidence—fingerprints. In recent years, legal experts have become deeply concerned about the accuracy of the “friction ridge analysis” central to fingerprint identification. Fingerprints are believed to be unique, but the process of matching prints has no statistically valid model. And forensic examiners are often working in an imperfect world, where prints taken in a police station on an ink pad are compared to prints left at a crime scene, which may be smudged or partially captured. Yet, as University of California–Los Angeles law professor Jennifer Mnookin has written, “fingerprint examiners typically testify in the language of absolute certainty.”

A 2006 study by the University of Southampton in England asked six veteran fingerprint examiners to study prints taken from actual criminal cases. The experts were not told that they had previously examined the same prints. The researchers’ goal was to determine if contextual information—for example, some prints included a notation that the suspect had already confessed—would affect the results. But the experiment revealed a far more serious problem: The analyses of fingerprint examiners were often inconsistent regardless of context. Only two of the six experts reached the same conclusions on second examination as they had on the first....

The double-helix structure of DNA was discovered in the 1950s, but it wasn’t until 30 years later that sample analysis became sophisticated enough for positive ID. In 1987, a serial rapist by the name of Tommie Lee Andrews was the first person convicted in the U.S. using DNA. Nevertheless, for several years scientists continued to research and debate what constitutes a satisfactory match. The resulting process is broadly accepted and quantifiable (when using the most advanced analysis, there is a one in more than a quadrillion chance of a random match of two strangers’ nuclear DNA).

But DNA constitutes less than 10 percent of the case load at U.S. crime labs. The goal going forward, everyone agrees, is to make the rest of forensics more rigorous and statistically grounded....

It will take years to fully reconcile the rigors of the scientific method with the needs and processes of the judicial system. But in the meantime, questionable forensic science will continue to tip the scales of justice. And when bad decisions are made in the courtroom, an innocent person’s entire life can be swept right out from under him. It happened to Steven Barnes 20 years ago. Then 23 years old, he was brought to trial for the rape and murder of a 16-year-old girl. He had never been arrested before and was confident he’d be cleared. Yet he watched as forensics expert Elaine Pagliaro testified that two hairs found in Barnes’s pickup were microscopically similar to the victim’s. Pagliaro also noted that soil samples taken from the truck were consistent with dirt from the crime scene and even that a distinctive pattern from the victim’s jeans was similar to an imprint left on the truck.

Due largely to her testimony, Barnes was sentenced to 25 years to life in prison. Last year, he was cleared by DNA and released. He’d never been on the Internet or used a cellular phone, and his girlfriend, who initially stuck by him after he went to prison, had long ago married another man. Barnes told Popular Mechanics that he works hard not to be overwhelmed by bitterness, even toward the jurors. “They must have thought, ‘[Pagliaro] knows what she is talking about.’”

Pagliaro, a veteran analyst with the Connecticut State Police, has recently co-authored a book called The Real World of a Forensic Scientist. “I think this scrutiny is actually good,” she says. “It’s important for the public to have a realistic expectation of what the science can do.” As for the Barnes case, there is no suggestion of impropriety regarding her testimony, but none of the evidence she presented was based on statistically validated science. “You feel awful someone spent all that time in jail,” she says. “All you can do is look back and say, ‘Was that the best we could do?’”

More here



(And don't forget your ration of Wicked Thoughts for today)