FrogHeart

The UK is apparently getting ready to introduce evidence-based science into government policies and decisions. According a Jan. 18, 2013 opinion piece by Michael Brooks for New Scientist, the Department for Education will be one of the first to institute this new approach (Note: Links have been removed),

It has been a long time coming, according to Chris Wormald, permanent secretary at the Department for Education. The civil service is not short of clever people, he points out, and there is no lack of desire to use evidence properly. More than 20 years as a serving politician has convinced him that they are as keen as anyone to create effective policies. “I’ve never met a minister who didn’t want to know what worked,” he says. What has changed now is that informed policy-making is at last becoming a practical possibility.

That is largely thanks to the abundance of accessible data and the ease with which new, relevant data can be created. This has supported a desire to move away from hunch-based politics. [emphasis miine]

Last week, for instance, Rebecca Endean, chief scientific advisor and director of analytical services at the Ministry of Justice, announced that the UK government is planning to open up its data for analysis by academics, accelerating the potential for use in policy planning.

At the same meeting, hosted by innovation-promoting charity NESTA, Wormald announced a plan to create teaching schools based on the model of teaching hospitals. In education, he said, the biggest single problem is a culture that often relies on anecdotal experience rather than systematically reported data from practitioners, as happens in medicine. [emphasis mine] “We want to move teacher training and research and practice much more onto the health model,” Wormald said.

Now what could possibly pose a problem for this charming idea of evidence-based policy planning?

In education, the evidence-based revolution has already begun. A charity called the Education Endowment Foundation is spending £1.4 million on a randomised controlled trial of reading programmes in 50 British schools.

There are reservations though. The Ministry of Justice is more circumspect about the role of such trials. Where it has carried out randomised controlled trials, they often failed to change policy, or even irked politicians with conclusions that were obvious. “It is not a panacea,” Endean says.

…

The biggest need is perhaps foresight. Ministers often need instant answers, and sometimes the data are simply not available. Bang goes any hope of evidence-based policy.

…

Crucial to the process will be convincing the public about the value and use of data, so that everyone is on-board. This is not going to be easy. When the government launched its Administrative Data Taskforce, which set out to look at data in all departments and opening it up so that it could be used for evidence-based policy, it attracted minimal media interest.

…

There are economic issues. Most of the predictable areas where data and evidence would be useful span different departments, and funding for research that involves multiple government departments is near-impossible to come by at the moment. “Only counter-terrorism gets cross-departmental funding,” Wiles says.

And those at the frontline of all this may also need convincing. Some teachers have already expressed reservations. There may be problems with parents not wanting their children to take part in education trials. For instance, in a control group they will feel left out of innovation; in the experimental arm they will worry that the old ways were better. What’s more, teachers may be tempted to halt a trial early if they feel it is not helping students. [emphasis mine]

There’s a basic assumption being made that evidence-based medicine has been a huge and howling success. I see no evidence cited either in this article or anywhere else that this has been the case. Medicine and health care research and practice have bifurcated in some fascinating ways. Researchers and clinicians live in different worlds and have very little contact. It is increasingly difficult (almost impossible) for a clinician to run a research project as the research process has become the province of professional grant writers and others who jump innumerable hoops for monies with the consequence that researchers have scant clinical experience.

No human clinical trial is ever large enough (1000 or 10000 or more) to give perfect insight into a drug’s effects once it gets used in the general population (millions and billions) which is why drugs that have managed to get approval are sometimes shown to be more dangerous and/or less effective than originally believed. There is also the problem of positive publication bias. Researchers publish studies that show positive results because journals don’t tend to print (by a wide margin) studies that have negative or inconclusive results. As for the vaunted evidence-based training process, doctors are learning to do less and less and becoming clerks who fill out referral forms and requests for tests that other professionals perform and interpret.

The loathing expressed towards hunches and anecdotes seems ironic since science is based on those two pillars. Is there a single story about a scientific breakthrough that doesn’t come down to a hunch or whatever you want to call that moment which sparked someone’s curiosity and suspicion? And, the reason for conferences is less about learned papers and more about informal conversations and meetings where people trade anecdotes.

Evidence-based science is a tool. It can be very powerful but it has to be used wisely and, in the case of those teachers who might want to withdraw students from a trial they thought was doing harm, that would be a very wise use of the evidence they’ve observed, wouldn’t it?

In the US, they’re also talking about science and politics but taking a very different perspective. Daniel Sarewitz, co-director of the Consortium for Science, Policy and Outcomes at Arizona State University based in Washington DC, has written a Jan. 2, 2013 article for Nature where he opines that scientists should not show political bias,

To prevent science from continuing its worrying slide towards politicization, here’s a New Year’s resolution for scientists, especially in the United States: gain the confidence of people and politicians across the political spectrum by demonstrating that science is bipartisan.

That President Barack Obama chose to mention “technology, discovery and innovation” in his passionate victory speech in November shows just how strongly science has come, over the past decade or so, to be a part of the identity of one political party, the Democrats, in the United States. The highest-profile voices in the scientific community have avidly pursued this embrace. For the third presidential election in a row, dozens of Nobel prizewinners in physics, chemistry and medicine signed a letter endorsing the Democratic candidate.

The 2012 letter argued that Obama would ensure progress on the economy, health and the environment by continuing “America’s proud legacy of discovery and invention”, and that his Republican opponent, Mitt Romney, would “devastate a long tradition of support for public research and investment in science”. The signatories wrote “as winners of the Nobel Prizes in Science”, thus cleansing their endorsement of the taint of partisanship by invoking their authority as pre-eminent scientists.

I think science groups/associations should follow the same policy a lot of business groups/organizations do, support/donate to all the viable parties and candidates but give more to your favourites. After all, you never know who’s going to be forming the next government so you want to keep in everyone’s good books.

It’s this suggestion which gives me pause,

If scientists want to claim that their recommendations are independent of their political beliefs, they ought to be able to show that those recommendations have the support of scientists with conflicting beliefs. Expert panels advising the government on politically divisive issues could strengthen their authority by demonstrating political diversity.

This rests on the notion that one’s affiliations are easily categorized. Not all Democrats are the same, nor are Republicans. While the parties are broadly Democrat and Republican within these parties there are subdivisions within subdivisions. It would likely cause more divisiveness than is worthwhile given the proposed benefit to attempt bipartisan panels, etc.

Daniel Lende in a Jan. 21, 2013 posting on the Neuroanthropology blog (member of the PLoS [Public Library of Science] blog network) comments at more length on the problems with Sarewitz’s thesis. For example,

… In Sarewitz’s world, politicians will naturally listen better to bipartisan endorsements. Take a recent example, the bipartisan Simpson-Bowles plan for “Financial Responsibility and Reform.” Politicians just fell over themselves to enact that bipartisan approach to dealing with a major social problem, didn’t they?

An even better example is smoking. If the model is somehow that by inserting better, more independent information into the political process, more rational and scientific decisions will be made, then that’s the wrong kind of scientific flow chart. Tobacco companies had major profits at stake, and major money to spend. They spent it on politicians and scientists alike. It wasn’t on a search for “the truth” about tobacco.

I’m not familiar enough with US politics to understand that reference to the ‘Simpson-Bowles plan’ but I take it that the bipartisan approach was a big failure in that instance. Lende also mounts an interesting discussion about social science and science as per some of Sarewitz’s comments about how the science community does not want to be perceived in the same light as the social science community (from Nature),

Conservatives in the US government have long been hostile to social science, which they believe tilts towards liberal political agendas. Consequently, the social sciences have remained poorly funded and politically vulnerable, and every so often Republicans threaten to eliminate the entire National Science Foundation budget for social science.

As scientists seek to provide policy-relevant knowledge on complex, interdisciplinary problems ranging from fisheries depletion and carbon emissions to obesity and natural hazards, the boundary between the natural and the social sciences has blurred more than many scientists want to acknowledge. With Republicans generally sceptical [sic] of government’s ability and authority to direct social and economic change, the enthusiasm with which leading scientists align themselves with the Democratic party can only reinforce conservative suspicions that for contentious issues such as climate change, natural-resource management and policies around reproduction, all science is social science.

Lende’s response echoes some of my own feelings on the topic still Sarewitz does have a point of sorts. Sarewitz dances between referring to scientists as individuals and in groups. As an individual, I think scientists should follow their own conscience with regard to whom and what they support but a science association or group should focus as much as possible on their pursuit of science irrespective of the politics. In short, you praise or condemn any and all governments according to what’s best for science.

Personally, I’m not a big fan of Canada’s Conservative party but this Conservative-led government has made some good choices, as well as, bad ones and I reserve the right to ‘praise them or bury them’ accordingly.