Tag Archives: S. James Gates

Scientific evidence and certainty: a controversy in the US Justice system

It seems that forensic evidence does not deliver the certainty that television and US prosecutors (I wonder if Canadian Crown Attorneys or Crown Counsels concur with their US colleagues?) would have us believe. The US President’s Council of Advisors on Science and Technology (PCAST) released a report (‘Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods‘ 174 pp PDF) on Sept. 20, 2016 that amongst other findings, notes that more scientific rigour needs to be applied to the field of forensic science.

Here’s more from the Sept. 20, 2016 posting by Eric Lander, William Press, S. James Gates, Jr., Susan L. Graham, J. Michael McQuade, and Daniel Schrag, on the White House blog,

The study that led to the report was a response to the President’s question to his PCAST in 2015, as to whether there are additional steps on the scientific side, beyond those already taken by the Administration in the aftermath of a highly critical 2009 National Research Council report on the state of the forensic sciences, that could help ensure the validity of forensic evidence used in the Nation’s legal system.

PCAST concluded that two important gaps warranted the group’s attention: (1) the need for clarity about the scientific standards for the validity and reliability of forensic methods and (2) the need to evaluate specific forensic methods to determine whether they have been scientifically established to be valid and reliable. The study aimed to help close these gaps for a number of forensic “feature-comparison” methods—specifically, methods for comparing DNA samples, bitemarks, latent fingerprints, firearm marks, footwear, and hair.

In the course of its year-long study, PCAST compiled and reviewed a set of more than 2,000 papers from various sources, educated itself on factual matters relating to the interaction between science and the law, and obtained input from forensic scientists and practitioners, judges, prosecutors, defense attorneys, academic researchers, criminal-justice-reform advocates, and representatives of Federal agencies.

A Sept. 23, 2016 article by Daniel Denvir for Salon.com sums up the responses from some of the institutions affected by this report,

Under fire yet again, law enforcement is fighting back. Facing heavy criticism for misconduct and abuse, prosecutors are protesting a new report from President Obama’s top scientific advisors that documents what has long been clear: much of the forensic evidence used to win convictions, including complex DNA samples and bite mark analysis, is not backed up by credible scientific research.

Although the evidence of this is clear, many in law enforcement seem terrified that keeping pseudoscience out of prosecutions will make them unwinnable. Attorney General Loretta Lynch declined to accept the report’s recommendations on the admissibility of evidence and the FBI accused the advisors of making “broad, unsupported assertions.” But the National District Attorneys Association, which represents roughly 2,5000 top prosecutors nationwide, went the furthest, taking it upon itself to, in its own words, “slam” the report.

Prosecutors’ actual problem with the report, produced by some of the nation’s leading scientists on the President’s Council of Advisors on Science and Technology, seems to be unrelated to science. Reached by phone NDAA president-elect Michael O. Freeman could not point to any specific problem with the research and accused the scientists of having an agenda against law enforcement.

“I’m a prosecutor and not a scientist,” Freeman, the County Attorney in Hennepin County, Minnesota, which encompasses Minneapolis, told Salon. “We think that there’s particular bias that exists in the folks who worked on this, and they were being highly critical of the forensic disciplines that we use in investigating and prosecuting cases.”

That response, devoid of any reference to hard science, has prompted some mockery, including from Robert Smith, Senior Research Fellow and Director of the Fair Punishment Project at Harvard Law School, who accused the NDAA of “fighting to turn America’s prosecutors into the Anti-Vaxxers, the Phrenologists, the Earth-Is-Flat Evangelists of the criminal justice world.”

It has also, however, also lent credence to a longstanding criticism that American prosecutors are more concerned with winning than in establishing a defendant’s guilt beyond a reasonable doubt.

“Prosecutors should not be concerned principally with convictions; they should be concerned with justice,” said Daniel S. Medwed, author of “Prosecution Complex: America’s Race to Convict and Its Impact on the Innocent” and a professor at Northern University School of Law, told Salon. “Using dodgy science to obtain convictions does not advance justice.”

Denvir’s article is lengthier and worth reading in its entirety.

Assuming there’s an association of forensic scientists, I find it interesting they don’t appear to have responded.

Finally, if there’s one thing you learn while writing about science it’s that there is no real certainty. For example, if you read about the Higgs boson discovery, you’ll note that the scientists involved the research never stated with absolute certainty that it exists but rather they ‘were pretty darn sure’ it does (I believe the scientific term is 5-sigma). There’s more about the Higgs boson and 5-sigma in this July 17, 2012 article by Evelyn Lamb for Scientific American,

In short, five-sigma corresponds to a p-value, or probability, of 3×10-7, or about 1 in 3.5 million. This is not the probability that the Higgs boson does or doesn’t exist; rather, it is the probability that if the particle does not exist, the data that CERN [European Particle Physics Laboratory] scientists collected in Geneva, Switzerland, would be at least as extreme as what they observed. “The reason that it’s so annoying is that people want to hear declarative statements, like ‘The probability that there’s a Higgs is 99.9 percent,’ but the real statement has an ‘if’ in there. There’s a conditional. There’s no way to remove the conditional,” says Kyle Cranmer, a physicist at New York University and member of the ATLAS team, one of the two groups that announced the new particle results in Geneva on July 4 [2012].

For the interested, there’s a lot more to Lamb’s article.

Getting back to forensic science, this PCAST report looks like an attempt to bring forensics back into line with the rest of the science world.

D-PLACE: an open access database of places, language, culture, and enviroment

In an attempt to be a bit more broad in my interpretation of the ‘society’ part of my commentary I’m including this July 8, 2016 news item on ScienceDaily (Note: A link has been removed),

An international team of researchers has developed a website at d-place.org to help answer long-standing questions about the forces that shaped human cultural diversity.

D-PLACE — the Database of Places, Language, Culture and Environment — is an expandable, open access database that brings together a dispersed body of information on the language, geography, culture and environment of more than 1,400 human societies. It comprises information mainly on pre-industrial societies that were described by ethnographers in the 19th and early 20th centuries.

A July 8, 2016 University of Toronto news release (also on EurekAlert), which originated the news item, expands on the theme,

“Human cultural diversity is expressed in numerous ways: from the foods we eat and the houses we build, to our religious practices and political organisation, to who we marry and the types of games we teach our children,” said Kathryn Kirby, a postdoctoral fellow in the Departments of Ecology & Evolutionary Biology and Geography at the University of Toronto and lead author of the study. “Cultural practices vary across space and time, but the factors and processes that drive cultural change and shape patterns of diversity remain largely unknown.

“D-PLACE will enable a whole new generation of scholars to answer these long-standing questions about the forces that have shaped human cultural diversity.”

Co-author Fiona Jordan, senior lecturer in anthropology at the University of Bristol and one of the project leads said, “Comparative research is critical for understanding the processes behind cultural diversity. Over a century of anthropological research around the globe has given us a rich resource for understanding the diversity of humanity – but bringing different resources and datasets together has been a huge challenge in the past.

“We’ve drawn on the emerging big data sets from ecology, and combined these with cultural and linguistic data so researchers can visualise diversity at a glance, and download data to analyse in their own projects.”

D-PLACE allows users to search by cultural practice (e.g., monogamy vs. polygamy), environmental variable (e.g. elevation, mean annual temperature), language family (e.g. Indo-European, Austronesian), or region (e.g. Siberia). The search results can be displayed on a map, a language tree or in a table, and can also be downloaded for further analysis.

It aims to enable researchers to investigate the extent to which patterns in cultural diversity are shaped by different forces, including shared history, demographics, migration/diffusion, cultural innovations, and environmental and ecological conditions.

D-PLACE was developed by an international team of scientists interested in cross-cultural research. It includes researchers from Max Planck Institute for the Science of Human history in Jena Germany, University of Auckland, Colorado State University, University of Toronto, University of Bristol, Yale, Human Relations Area Files, Washington University in Saint Louis, University of Michigan, American Museum of Natural History, and City University of New York.

The diverse team included: linguists; anthropologists; biogeographers; data scientists; ethnobiologists; and evolutionary ecologists, who employ a variety of research methods including field-based primary data collection; compilation of cross-cultural data sources; and analyses of existing cross-cultural datasets.

“The team’s diversity is reflected in D-PLACE, which is designed to appeal to a broad user base,” said Kirby. “Envisioned users range from members of the public world-wide interested in comparing their cultural practices with those of other groups, to cross-cultural researchers interested in pushing the boundaries of existing research into the drivers of cultural change.”

Here’s a link to and a citation for the paper,

D-PLACE: A Global Database of Cultural, Linguistic and Environmental Diversity by Kathryn R. Kirby, Russell D. Gray, Simon J. Greenhill, Fiona M. Jordan, Stephanie Gomes-Ng, Hans-Jörg Bibiko, Damián E. Blasi, Carlos A. Botero, Claire Bowern, Carol R. Ember, Dan Leehr, Bobbi S. Low, Joe McCarter, William Divale, Michael C. Gavin.  PLOS ONE, 2016; 11 (7): e0158391 DOI: 10.1371/journal.pone.0158391 Published July 8, 2016.

This paper is open access.

You can find D-PLACE here.

While it might not seem like that there would be a close link between anthropology and physics in the 19th and early 20th centuries, that information can be mined for more contemporary applications. For example, someone who wants to make a case for a more diverse scientific community may want to develop a social science approach to the discussion. The situation in my June 16, 2016 post titled: Science literacy, science advice, the US Supreme Court, and Britain’s House of Commons, could  be extended into a discussion and educational process using data from D-Place and other sources to make the point,

Science literacy may not be just for the public, it would seem that US Supreme Court judges may not have a basic understanding of how science works. David Bruggeman’s March 24, 2016 posting (on his Pasco Phronesis blog) describes a then current case before the Supreme Court (Justice Antonin Scalia has since died), Note: Links have been removed,

It’s a case concerning aspects of the University of Texas admissions process for undergraduates and the case is seen as a possible means of restricting race-based considerations for admission.  While I think the arguments in the case will likely revolve around factors far removed from science and or technology, there were comments raised by two Justices that struck a nerve with many scientists and engineers.

Both Justice Antonin Scalia and Chief Justice John Roberts raised questions about the validity of having diversity where science and scientists are concerned [emphasis mine].  Justice Scalia seemed to imply that diversity wasn’t esential for the University of Texas as most African-American scientists didn’t come from schools at the level of the University of Texas (considered the best university in Texas).  Chief Justice Roberts was a bit more plain about not understanding the benefits of diversity.  He stated, “What unique perspective does a black student bring to a class in physics?”

To that end, Dr. S. James Gates, theoretical physicist at the University of Maryland, and member of the President’s Council of Advisers on Science and Technology (and commercial actor) has an editorial in the March 25 [2016] issue of Science explaining that the value of having diversity in science does not accrue *just* to those who are underrepresented.

Dr. Gates relates his personal experience as a researcher and teacher of how people’s background inform their practice of science, and that two different people may use the same scientific method, but think about the problem differently.

I’m guessing that both Scalia and Roberts and possibly others believe that science is the discovery and accumulation of facts. In this worldview science facts such as gravity are waiting for discovery and formulation into a ‘law’. They do not recognize that most science is a collection of beliefs and may be influenced by personal beliefs. For example, we believe we’ve proved the existence of the Higgs boson but no one associated with the research has ever stated unequivocally that it exists.

More generally, with D-PLACE and the recently announced Trans-Atlantic Platform (see my July 15, 2016 post about it), it seems Canada’s humanities and social sciences communities are taking strides toward greater international collaboration and a more profound investment in digital scholarship.

Science literacy, science advice, the US Supreme Court, and Britain’s House of Commons

This ‘think’ piece is going to cover a fair bit of ground including science literacy in the general public and in the US Supreme Court, and what that might mean for science advice and UK Members of Parliament (MPs).

Science literacy generally and in the US Supreme Court

A science literacy report for the US National Academy of Sciences (NAS), due sometime from early to mid 2017, is being crafted with an eye to capturing a different perspective according to a March 24, 2016 University of Wisconsin-Madison news release by Terry Dewitt,

What does it mean to be science literate? How science literate is the American public? How do we stack up against other countries? What are the civic implications of a public with limited knowledge of science and how it works? How is science literacy measured?

These and other questions are under the microscope of a 12-member National Academy of Sciences (NAS) panel — including University of Wisconsin—Madison Life Sciences Communication Professor Dominique Brossard and School of Education Professor Noah Feinstein — charged with sorting through the existing data on American science and health literacy and exploring the association between knowledge of science and public perception of and support for science.

The committee — composed of educators, scientists, physicians and social scientists — will take a hard look at the existing data on the state of U.S. science literacy, the questions asked, and the methods used to measure what Americans know and don’t know about science and how that knowledge has changed over time. Critically for science, the panel will explore whether a lack of science literacy is associated with decreased public support for science or research.

Historically, policymakers and leaders in the scientific community have fretted over a perceived lack of knowledge among Americans about science and how it works. A prevailing fear is that an American public unequipped to come to terms with modern science will ultimately have serious economic, security and civic consequences, especially when it comes to addressing complex and nuanced issues like climate change, antibiotic resistance, emerging diseases, environment and energy choices.

While the prevailing wisdom, inspired by past studies, is that Americans don’t stack up well in terms of understanding science, Brossard is not so convinced. Much depends on what kinds of questions are asked, how they are asked, and how the data is analyzed.

It is very easy, she argues, to do bad social science and past studies may have measured the wrong things or otherwise created a perception about the state of U.S. science literacy that may or may not be true.

“How do you conceptualize scientific literacy? What do people need to know? Some argue that scientific literacy may be as simple as an understanding of how science works, the nature of science, [emphasis mine]” Brossard explains. “For others it may be a kind of ‘civic science literacy,’ where people have enough knowledge to be informed and make good decisions in a civics context.”

Science literacy may not be just for the public, it would seem that US Supreme Court judges may not have a basic understanding of how science works. David Bruggeman’s March 24, 2016 posting (on his Pasco Phronesis blog) describes a then current case before the Supreme Court (Justice Antonin Scalia has since died), Note: Links have been removed,

It’s a case concerning aspects of the University of Texas admissions process for undergraduates and the case is seen as a possible means of restricting race-based considerations for admission.  While I think the arguments in the case will likely revolve around factors far removed from science and or technology, there were comments raised by two Justices that struck a nerve with many scientists and engineers.

Both Justice Antonin Scalia and Chief Justice John Roberts raised questions about the validity of having diversity where science and scientists are concerned [emphasis mine].  Justice Scalia seemed to imply that diversity wasn’t esential for the University of Texas as most African-American scientists didn’t come from schools at the level of the University of Texas (considered the best university in Texas).  Chief Justice Roberts was a bit more plain about not understanding the benefits of diversity.  He stated, “What unique perspective does a black student bring to a class in physics?”

To that end, Dr. S. James Gates, theoretical physicist at the University of Maryland, and member of the President’s Council of Advisers on Science and Technology (and commercial actor) has an editorial in the March 25 [2016] issue of Science explaining that the value of having diversity in science does not accrue *just* to those who are underrepresented.

Dr. Gates relates his personal experience as a researcher and teacher of how people’s background inform their practice of science, and that two different people may use the same scientific method, but think about the problem differently.

I’m guessing that both Scalia and Roberts and possibly others believe that science is the discovery and accumulation of facts. In this worldview science facts such as gravity are waiting for discovery and formulation into a ‘law’. They do not recognize that most science is a collection of beliefs and may be influenced by personal beliefs. For example, we believe we’ve proved the existence of the Higgs boson but no one associated with the research has ever stated unequivocally that it exists.

For judges who are under the impression that scientific facts are out there somewhere waiting to be discovered diversity must seem irrelevant. It is not. Who you are affects the questions you ask and how you approach science. The easiest example is to look at how women were viewed when they were subjects in medical research. The fact that women’s physiology is significantly different (and not just in child-bearing ways) was never considered relevant when reporting results. Today, researchers consider not only gender, but age (to some extent), ethnicity, and more when examining results. It’s still not a perfect but it was a step forward.

So when Brossard included “… an understanding of how science works, the nature of science …” as an aspect of science literacy, the judges seemed to present a good example of how not understanding science can have a major impact on how others live.

I’d almost forgotten this science literacy piece as I’d started the draft some months ago but then I spotted a news item about a science advice/MP ‘dating’ service in the UK.

Science advice and UK MPs

First, the news, then, the speculation (from a June 6, 2016 news item on ScienceDaily),

MPs have expressed an overwhelming willingness to use a proposed new service to swiftly link them with academics in relevant areas to help ensure policy is based on the latest evidence.

A June 6, 2016 University of Exeter press release, which originated the news item, provides more detail about the proposed service and the research providing the supporting evidence (Note: A link has been removed),

The government is pursuing a drive towards evidence-based policy, yet policy makers still struggle to incorporate evidence into their decisions. One reason for this is limited easy access to the latest research findings or to academic experts who can respond to questions about evidence quickly.

Researchers at Cardiff University, the University of Exeter and University College London have today published results of the largest study to date reporting MPs’ attitudes to evidence in policy making and their reactions to a proposed Evidence Information Service (EIS) – a rapid match-making advisory service that would work alongside existing systems to put MPs in touch with relevant academic experts.

Dr Natalia Lawrence, of the University of Exeter, said: “It’s clear from our study that politicians want to ensure their decisions incorporate the most reliable evidence, but it can sometimes be very difficult for them to know how to access the latest research findings. This new matchmaking service could be a quick and easy way for them to seek advice from cutting-edge researchers and to check their understanding and facts. It could provide a useful complement to existing highly-valued information services.”

The research, published today in the journal Evidence and Policy, reports the findings of a national consultation exercise between politicians and the public. The researchers recruited members of the public to interview their local parliamentary representative. In total 86, politicians were contacted with 56 interviews completed. The MPs indicated an overwhelming willingness to use a service such as the EIS, with 85% supporting the idea, but noted a number of potential reservations related to the logistics of the EIS such as response time and familiarity with the service. Yet, the MPs indicated that their logistical reservations could be overcome by accessing the EIS via existing highly-valued parliamentary information services such as those provided by the House of Commons and Lords Libraries. Furthermore prior to rolling out the EIS on a nationwide basis it would first need to be piloted.

Developing the proposed EIS in line with feedback from this consultation of MPs would offer the potential to provide policy makers with rapid, reliable and confidential evidence from willing volunteers from the research community.

Professor Chris Chambers, of Cardiff University, said: “The government has given a robust steer that MPs need to link in more with academics to ensure decisions shaping the future of the country are evidence-based. It’s heartening to see that there is a will to adopt this system and we now need to move into a phase of developing a service that is both simple and effective to meet this need.”

The next steps for the project are parallel consultations of academics and members of the public and a pilot of the EIS, using funding from GW4 alliance of universities, made up of Bath, Bristol, Cardiff and Exeter.

What this study shows:
• The consultation shows that politicians recognise the importance of evidence-based policy making and agree on the need for an easier and more direct linkage between academic experts and policy makers.
• Politicians would welcome the creation of the EIS as a provider of rapid, reliable and confidential evidence.

What this study does not show:
• This study does not show how academics would provide evidence. This was a small-scale study which consulted politicians and has not attempted to give voice to the academic community.
• This study does not detail the mechanism of an operational EIS. Instead it indicates the need for a service such as the EIS and suggests ways in which the EIS can be operationalized.

Here’s a link to and a citation for the paper,

Service as a new platform for supporting evidence-based policy: a consultation of UK parliamentarians by Natalia Lawrence, Jemma Chambers, Sinead Morrison, Sven Bestmann, Gerard O’Grady, Christopher Chambers, Andrew Kythreotis. Evidence & Policy: A Journal of Research, Debate and Practice DOI: http://dx.doi.org/10.1332/174426416X14643531912169 Appeared or available online: June 6, 2016

This paper is behind a paywall open access. *Corrected June 17, 2016.*

It’s an interesting idea and I can understand the appeal. However, operationalizing this ‘dating’ or ‘matchmaking’ service could prove quite complex. I appreciate the logistics issues but I’m a little more concerned about the MPs’ science literacy. Are they going to be like the two US justices who believe that science is the pursuit of immutable facts? What happens if two MPs are matched up with a different scientist and those two scientists didn’t agree about what the evidence says. Or, what happens if one scientist is more cautious than the other. There are all kinds of pitfalls. I’m not arguing against the idea but it’s going to require a lot of careful consideration.