I made a mistake when reporting on NASA and the ‘arsenic’ bacterium. Apparently, the research methodology was problematic and the conclusion that the bacterium can substitute arsenic for phosphorus in its DNA is not supported by the evidence as presented.
Martin Robbins at the Lay Scientist blog (one of The Guardian’s science blogs) has posted an analysis of how this ‘media storm’ occurred. The article which started it all was in a well respected, peer-reviewed journal, Science (which is published by the American Association for the Advancement of Science). From Robbins’s Dec. 8, 2010 posting,
Should the paper have been published in the first place? Carl Zimmer’s blog post for Slate collects the responses of numerous scientists to the work, including the University of Colorado’s Shelley Copley declaring that: “This paper should not have been published.”
There are two distinct questions here to tease apart: ‘should the paper have been published?’ and ‘should it have been published in Science?’
To the first question I would say ‘yes’. Peer review isn’t supposed to be about declaring whether a paper is definitely right and therefore fit for publication on that basis. The purpose of publishing paper is to submit ideas for further discussion and debate, with peer review serving as a fairly loose filter to weed out some of the utter crap. The contribution a paper makes to science goes far beyond such trivialities as whether or not it’s actually right.
Wolfe-Simon et al’s paper might be wrong, but it has also sparked an interesting and useful debate on the evidence and methodology required to make claims about this sort of thing, and the next paper on this subject that comes along with hopefully be a lot stronger as a result of this public criticism. You could argue on that basis that its publication is useful.
I would argue that the real bone of contention is whether it should have been published in Science – after all, if it had appeared in the Journal of Speculative Biological Hypotheses (and not been hyped) nobody would have given a crap. On this I’m not really qualified to comment, but what I can say is that given the wealth of scientists coming forward to criticize the work, it’s remarkable that the journal found three willing to pass it.
Robbins goes on to analyze the impact that the embargo (story is considered confidential until a prescribed date) that Science applied to the story about the article had on mainstream and other media. He also notes the impact that bloggers had on the story,
The quality, accuracy and context of material available on leading blogs exceeded that of much of mainstream media reporting by light years. While newspapers ran away with the story, it was left to bloggers like Ed Yong, Carl Zimmer, Lewis Dartnell and Phil Plait to put things into perspective.
But more importantly it turns out that peer review is being done on blogs. John Hawks and Alex Bradley – both scientists with relevant expertise – found methodological problems. Rosie Redfield, a microbiology professor a the University of British Colombia [sic], wrote an extensive and detailed take-down of the paper on her blog that morphed into a letter to Science, which I sincerely hope they publish.
Robbins does not suggest that the blogosophere is the perfect place for peer review only that it played an important role regarding this research. There is much more to the posting and I do encourage you to read it.
I did look at Rosie Redfield’s postings about the papers. I found her Dec. 4, 2010 posting to provide the most accessible analysis of the methodological issues of the two. Her Dec. 8, 2010 posting is her submission to Science about the matter.
I do apologize for getting caught up in the frenzy.