Tag Archives: Daniel Schrag

Scientific evidence and certainty: a controversy in the US Justice system

It seems that forensic evidence does not deliver the certainty that television and US prosecutors (I wonder if Canadian Crown Attorneys or Crown Counsels concur with their US colleagues?) would have us believe. The US President’s Council of Advisors on Science and Technology (PCAST) released a report (‘Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods‘ 174 pp PDF) on Sept. 20, 2016 that amongst other findings, notes that more scientific rigour needs to be applied to the field of forensic science.

Here’s more from the Sept. 20, 2016 posting by Eric Lander, William Press, S. James Gates, Jr., Susan L. Graham, J. Michael McQuade, and Daniel Schrag, on the White House blog,

The study that led to the report was a response to the President’s question to his PCAST in 2015, as to whether there are additional steps on the scientific side, beyond those already taken by the Administration in the aftermath of a highly critical 2009 National Research Council report on the state of the forensic sciences, that could help ensure the validity of forensic evidence used in the Nation’s legal system.

PCAST concluded that two important gaps warranted the group’s attention: (1) the need for clarity about the scientific standards for the validity and reliability of forensic methods and (2) the need to evaluate specific forensic methods to determine whether they have been scientifically established to be valid and reliable. The study aimed to help close these gaps for a number of forensic “feature-comparison” methods—specifically, methods for comparing DNA samples, bitemarks, latent fingerprints, firearm marks, footwear, and hair.

In the course of its year-long study, PCAST compiled and reviewed a set of more than 2,000 papers from various sources, educated itself on factual matters relating to the interaction between science and the law, and obtained input from forensic scientists and practitioners, judges, prosecutors, defense attorneys, academic researchers, criminal-justice-reform advocates, and representatives of Federal agencies.

A Sept. 23, 2016 article by Daniel Denvir for Salon.com sums up the responses from some of the institutions affected by this report,

Under fire yet again, law enforcement is fighting back. Facing heavy criticism for misconduct and abuse, prosecutors are protesting a new report from President Obama’s top scientific advisors that documents what has long been clear: much of the forensic evidence used to win convictions, including complex DNA samples and bite mark analysis, is not backed up by credible scientific research.

Although the evidence of this is clear, many in law enforcement seem terrified that keeping pseudoscience out of prosecutions will make them unwinnable. Attorney General Loretta Lynch declined to accept the report’s recommendations on the admissibility of evidence and the FBI accused the advisors of making “broad, unsupported assertions.” But the National District Attorneys Association, which represents roughly 2,5000 top prosecutors nationwide, went the furthest, taking it upon itself to, in its own words, “slam” the report.

Prosecutors’ actual problem with the report, produced by some of the nation’s leading scientists on the President’s Council of Advisors on Science and Technology, seems to be unrelated to science. Reached by phone NDAA president-elect Michael O. Freeman could not point to any specific problem with the research and accused the scientists of having an agenda against law enforcement.

“I’m a prosecutor and not a scientist,” Freeman, the County Attorney in Hennepin County, Minnesota, which encompasses Minneapolis, told Salon. “We think that there’s particular bias that exists in the folks who worked on this, and they were being highly critical of the forensic disciplines that we use in investigating and prosecuting cases.”

That response, devoid of any reference to hard science, has prompted some mockery, including from Robert Smith, Senior Research Fellow and Director of the Fair Punishment Project at Harvard Law School, who accused the NDAA of “fighting to turn America’s prosecutors into the Anti-Vaxxers, the Phrenologists, the Earth-Is-Flat Evangelists of the criminal justice world.”

It has also, however, also lent credence to a longstanding criticism that American prosecutors are more concerned with winning than in establishing a defendant’s guilt beyond a reasonable doubt.

“Prosecutors should not be concerned principally with convictions; they should be concerned with justice,” said Daniel S. Medwed, author of “Prosecution Complex: America’s Race to Convict and Its Impact on the Innocent” and a professor at Northern University School of Law, told Salon. “Using dodgy science to obtain convictions does not advance justice.”

Denvir’s article is lengthier and worth reading in its entirety.

Assuming there’s an association of forensic scientists, I find it interesting they don’t appear to have responded.

Finally, if there’s one thing you learn while writing about science it’s that there is no real certainty. For example, if you read about the Higgs boson discovery, you’ll note that the scientists involved the research never stated with absolute certainty that it exists but rather they ‘were pretty darn sure’ it does (I believe the scientific term is 5-sigma). There’s more about the Higgs boson and 5-sigma in this July 17, 2012 article by Evelyn Lamb for Scientific American,

In short, five-sigma corresponds to a p-value, or probability, of 3×10-7, or about 1 in 3.5 million. This is not the probability that the Higgs boson does or doesn’t exist; rather, it is the probability that if the particle does not exist, the data that CERN [European Particle Physics Laboratory] scientists collected in Geneva, Switzerland, would be at least as extreme as what they observed. “The reason that it’s so annoying is that people want to hear declarative statements, like ‘The probability that there’s a Higgs is 99.9 percent,’ but the real statement has an ‘if’ in there. There’s a conditional. There’s no way to remove the conditional,” says Kyle Cranmer, a physicist at New York University and member of the ATLAS team, one of the two groups that announced the new particle results in Geneva on July 4 [2012].

For the interested, there’s a lot more to Lamb’s article.

Getting back to forensic science, this PCAST report looks like an attempt to bring forensics back into line with the rest of the science world.