Popular Science set off a ‘storm of talk’ with its decision to shut down its online Comments section citing evidence-based science to support the decision. From the Sept. 24, 2013 posting by Suzanne LaBarre [Online Content Director] on Popular Science,
Comments can be bad for science. That’s why, here at PopularScience.com, we’re shutting them off.
…
But even a fractious minority wields enough power to skew a reader’s perception of a story, recent research suggests. In one study led by University of Wisconsin-Madison professor Dominique Brossard, 1,183 Americans read a fake blog post on nanotechnology and revealed in survey questions how they felt about the subject (are they wary of the benefits or supportive?). Then, through a randomly assigned condition, they read either epithet- and insult-laden comments (“If you don’t see the benefits of using nanotechnology in these kinds of products, you’re an idiot” ) or civil comments.
I’ll get back to the study led by Brossard but first, here are some of the reactions:
Will Oremus writing for Slate writes this on Sept. 24, 2013,
“Popular Science announced today that it is shutting off comments for good. “Comments can be bad for science,” explained Suzanne LaBarre, the magazine’s online content director. She pointed to “trolls and spambots” who “overwhelm” the magazine’s efforts to “spread the word of science far and wide.” Thanks to these trolls, she went on, “the cynical work of undermining bedrock scientific doctrine is now being done beneath our own stories, within a website devoted to championing science.”
Now, I get as annoyed as the next right-thinking person when Internet commenters misconstrue scientific research—let alone when they regale me with tales of their aunt’s third cousin who makes $73 an hour working from home. But I couldn’t help but notice an almost religious zeal in LaBarre’s framing of her magazine’s mission. Spreading the word of science? Undermining bedrock scientific doctrine? Substitute “Christianity” for “science” and “Christian” for “scientific” in those two phrases and perhaps you’ll see what makes me uncomfortable here. These aren’t the words of a scientist. They’re the words of an evangelist.
Sure, some very important scientific questions are pretty much settled, and it’s journalists’ job to convey that. But LaBarre’s metaphors conjure an image of science as an ancient and immovable stone fortress, from which the anointed few (Popular Science staff writers, say) may cast pearls in the direction of the masses below, but which might crumble to dust if the teeming throngs aren’t kept at bay. This conception is antithetical to the spirit of free inquiry that has always driven scientific discovery.
Oremus goes on to discuss the importance of comments both generally and specifically with regard to science. He also mentions support for this action although it is qualified in a Sept. 24, 2013 piece by Derek Thompson for Atlantic (Note: A link has been removed),
I’ve heard comment sections compared to “sewers”, and that sounds rather literally correct. Most of them are logistically required, but consistently disgusting, subterranean conduits for what is, technically speaking, waste. Not The Atlantic, of course. We have the Internet’s best commenters. But, you know, other people’s websites.
Are comments good or bad for online journalism? The best answer is the least satisfying. They are both. Some articles are better with feedback. Some articles work better with a quiet audience. Some sites channel conversation to become as delightful as the paid-writer paragraphs. Some sites don’t. Some commenters are wonderful teachers, even sources (I can’t tell you how much I’ve learned from my own). Many commenters are contemptible trolls.
A few months ago, The Atlantic digital team convened a brief meeting about our own comment strategy. After hours of meetings, most of us decided we wanted a comment section that felt democratic, encouraged voting, prized smarts, and included fail-safes for deleting racism, sexism, and the worst kinds of deliberate idiocy. The fact is, we want to hear from you. You make The Atlantic a better site. …
A Sept. 25, 2013 piece on BBC (British Broadcasting Corporation) News online relates the Popular Science action to online bullying and recent incidents in the UK,
Trolls – rude and uncivil online commentators – have been a growing problem for online publishers.
…
Micro-blogging site Twitter has come in for criticism for not doing more to combat trolls using its service.
Last week, the Metropolitan Police warned that cases of online abuse were on the rise.
The number of incidents reported to London police relating to online harassment and bullying now approached 2,000 each year, it said.
Earlier this year Caroline Criado-Perez faced abuse and threats of rape from emailers and Twitter trolls after successfully campaigning for a woman [writer Jane Austen] to be featured on UK banknotes.
Twitter has recently introduced a “report tweet” button to try to combat trolling.
Mathew Ingram in a Sept. 24, 2013 piece for paidcontent.org says (Note: Links have been removed),
There’s been a lot of sound and fury around reader comments lately: Gawker Media founder Nick Denton is rolling out a new platform that he hopes will help improve them, the New York Times magazine ran a long piece that tried to analyze how and why they got so bad, and now Popular Science magazine has decided to shut off comments altogether because they are apparently “bad for science.”
I’m tempted to argue that it’s also bad for science when you jump to conclusions based on very little evidence, or when you close off potential avenues for informed debate that might help your reporting, but there’s a bit more to it.
…
Presumably, fighting the trolls and spam — which almost every internet site, including this one, has to deal with as a matter of course — sucked up valuable resources that Popular Science believes could be better used for actually writing about science rather than trying to moderate comments, as a number of defenders of the magazine’s decision argued.
Chris Mooney writing for Mother Jones wrote about the study and comments in a Jan. 10, 2013 article where he seems to support the notion of stifling comments,
The study did not examine online climate change trolls directly—but there is good reason to think that the effects of their obnoxious behavior will, if anything, be worse. As the researchers note in the paper, compared with climate change, relatively few people know much about or have strong feelings about nanotechnology. When it comes to climate change, in contrast, “the controversy that you see in comments falls on more fertile ground, and resonates more with an established set of values that the reader may bring to the table,” explains study coauthor Dietram Scheufele, a professor of science communication at the University of Wisconsin-Madison. If commenters have stronger emotions and more of a stake, it stands to reason that the polarizing effect of their insults may be even stronger—although, to be sure, this needs to be studied.
…
To be sure, we all retain the option of not reading the comments. Which, in light of the latest research, is looking smarter than ever.
Now it’s time for the research. There were two pieces published one in January 2013, which was a perspective piece in Science magazine, and the other was a study published the Journal of Computer-Mediated Communication in Feb. 2013.
I featured the work Science magazine article in a Jan. 4, 2013 posting and so you can see for it yourself, here’s a citation and a link to the paper (which is behind a paywall),
Science, New Media, and the Public by Dominique Brossard and Dietram A. Scheufele in Science 4 January 2013: Vol. 339 no. 6115 pp. 40-41 DOI: 10.1126/science.1232329
From the paper (Note: Footnote has been removed),
Nine in 10 internet users in the United States turn to search engines to find information, and 60% of the U.S. public seeking information about specific scientific issues lists the Internet as their primary source of information.
Here’s the good side of science online (Note: A footnote has been removed),
Among the U.S. public, time spent on the World Wide Web has been linked to more positive attitudes toward science, even when controlling for use of traditional mass media such as newspapers and television. For instance, frequent Web users are more likely to report in surveys that they support basic scientific research even if it may not have immediate societal benefits. Research suggests that the availability of science news from the Internet may inform U.S. audiences with different educational backgrounds. In other words, online science sources may be helping to narrow knowledge gaps caused partly by science coverage in traditional media that tends to be tailored to highly educated audiences
The authors highlight three countervailing trends. In other words, there is a concerning side to all this lovely access to science on the internet. (1) The near demise of science in mainstream outlets. (2) The influence that search engines exert on the science queries people make. (3) The impact that comments can have on other readers’ perceptions and response to science information. From the paper,
A recent conference presented an examination of the effects of these unintended influences of Web 2.0 environments empirically by manipulating only the tone of the comments (civil or uncivil) that followed an online science news story in a national survey experiment. All participants were exposed to the same, balanced news item (covering nanotechnology as an emerging technology) and to a set of comments following the story that were consistent in terms of content but differed in tone. Disturbingly, readers’ interpretations of potential risks associated with the technology described in the news article differed significantly depending only on the tone of the manipulated reader comments posted with the story. Exposure to uncivil comments (which included name calling and other non–content-specific expressions of incivility) polarized the views among proponents and opponents of the technology with respect to its potential risks.
This notion that the comments are influencing readers’ perceptions is fascinating but there was no indication of just how large a sample was examined in the presentation the authors mention in this January 2013 piece. The conference presentation cited by the authors was a student one (from the Association for Education in Journalism and Mass Media; Communication Technology 2011 abstracts webpage),
Crude comments and concern: Online incivility’s effect on risk perceptions of emerging technologies • Peter Ladwig; Ashley Anderson, University of Wisconsin-Madison • Uncivil rhetoric has become a growing aspect of American political discussion and deliberation. This trend is not only confined to traditional media representations of deliberation, but also online media such as blog comments. This study examines online incivility’s effect on risk perception of an emerging technology, nanotechnology. We found that reading can polarize audiences’ attitudes of risk perception of nanotechnology along the lines of religiosity, efficacy, and support for the technology.
As you can see there’s not a lot of detail so it’s impossible to assess the research other than to note that is is suggestive. Arguably, the comments section reflects real life situations where people attend meetings or other events and are swayed by the speakers and whether or not they are experiencing a civil or uncivil environment.
My point here is that it’s good idea to read the research yourself and to check the sources. As it turns out a full study was published later,
The “Nasty Effect:” Online Incivility and Risk Perceptions of Emerging Technologies by Ashley A. Anderson, Dominique Brossard, Dietram A. Scheufele, Michael A. Xenos, Peter Ladwig. Journal of Computer-Mediated Communication doi: 10.1111/jcc4.12009
Unusually with a Wiley publication, this study appears to be open access.
While I find this an interesting and worthwhile study, it’s a little disappointing that LaBarre who’s representing Popular Science and its decision and others who cite the study don’t seem to have read it in its entirety, as there a few questions to be asked about the methodology and some of the inferences.
Participants (final sample size was 1,183) were asked to fill out a pretest survey about their media habits, knowledge of and support for nanotechnology, etc. then the researchers had participants read a neutral article (risks and benefits mentioned) on nanotechnology, from a Canadian newspaper, which was manipulated in one of eight ways to include civil comments and/or uncivil comments.
Here’s a source of some of my confusion about this study. From the paper’s description of the sample,
This study employed a nationally representative sample of the American population (N = 2,338) for an online survey with an embedded experiment conducted by Knowledge Networks with a completion rate of 54.2 percent.
Who was involved in this study? Someone willing to fill out a pretest survey and, by the way, it looks like they lost over 1,100 participants before the study was completed. As well, I don’t know what nationally representative means or how these people became part of the study. Did they answer an ad? How was participation solicited? Self-selection, which seems to be the case here, can have an impact on the results of your study.
As for the inferences LaBarre and others have made, the study despite the authors’ conclusions is not unequivocal as to the relationship between incivility and support for ‘science’ in this nanotechnology,
The purpose of this study was to explore online incivility’s role in polarizing attitudes when reading deliberation in a blog setting. We employed a topic with low familiarity among the general public, nanotechnology, and assessed formation of the perception of its risk in order to shed light on online incivility’s impact. The data reveal several important predictors of risk perception of nanotechnology as well as two significant interactions between civil or uncivil blog comments and value predispositions that individuals employ when processing information and making judgments about new technologies. Most importantly, this study found that uncivil blog comments contribute to polarization of risk perception of an issue depending on an individual’s level of religiosity and support of that entity. Specifically, among individuals who do not support nanotechnology, those who are exposed to uncivil deliberation in blog comments are more likely to perceive the technology as risky than those who are exposed to civil comments. Similarly, highly religious individuals are more likely to perceive nanotechnology as risky when exposed to uncivil comments compared to less religious individuals exposed to uncivil comments.
So someone who didn’t support nanotechnology in the first place was influenced by uncivil comments as was the person who is highly religious.
I think the researchers have jumped to a conclusion that is not entirely supported by their research, from the paper,
Online communication and discussion of new topics such as emerging technologies has the potential to enrich public deliberation. Nevertheless, this study’s findings show that online incivility may impede this democratic goal.
Let’s see if I can tie this all up with a bow, eh? An online community can be just as toxic as a real life community, which means that age old remedies may need to be employed along with remedies afforded by technology. Sometimes you have to shut down a debate, a public meeting, or, in this case, public comments, i.e., letting tempers cool. More that one of the commentators quoted here suggested that trying to find a solution to the problem is more helpful than simply shutting down and bowing out. I hope Popular Science is simply trying to cool the situation down while work is done back channel to develop a better way of handling comments.
As for the “democratic goal” of public deliberation, that’s never been easy and has often invited a great deal of incivility most of it not observed by researchers. The internet has allowed us to record the incivility but it is by no means new. For example,I once had the great good fortune to see some television clips of the institution of Saskatchewan’s medical system (precursor to Canada’s current system) and was shocked to see doctors in their suits and ties (it was in the early 1960s) screaming, shaking their fists, shoving, and fighting in protest of the first publicly funded medical care system in Canada. For more see Wikipedia’s Saskatchewan Doctors’ Strike.
ETA Sept. 27, 2013: My comment about researchers not being able to observe incivility should have included the phrase “with such ease as the internet affords us.”