Let’s be pragmatic about statistical mistakes

Tom Sigfried opened his talk with, “It’s time to stir things up.” Balance, he said, is a bad idea: it’s a journalist’s responsibility to weigh the evidence and draw a conclusion. Merely citing sources leads to the implicit equation of good science with bad or pseudo- science. Science writers should strive for reliable information above all else. That necessarily involves a deep knowledge of the field.

Which raises the question:
Whose responsibility is it to make sure that the public gets reliable information?

There certainly should be accountability at every level for mistakes made. But if the public at large, undergrad psychology majors, and in many cases the scientists themselves can’t grasp statistics, how can journalists be expected to catch everyone else’s oversights?

Practically speaking, the battle to educate science writers to evaluate all the evidence themselves may not be worth fighting. Asking journalists to know more than scientists about their topics is certainly unreasonable. In some sense, they have to be able to trust their sources, or at least the consensus among many of their sources.

Adam Frank noted that it must be difficult for journalists to know which scientists to trust. It takes scientists themselves a while to learn who rushes to publication and whose work is consistently careful and good.

Alexandra Witze rejoined that there are at least a couple of areas in which journalists can and ought to have a couple of trusted sources (climate change, stem cell research).

Charles Petit pointed out that these problems with properly weighing the evidence provide journalists with a chance to emphasize the self-corrective nature of science. Science journalists should embrace botched articles as a chance to tell people what science is all about: finding and fixing mistakes.

Internet Presents Problems and Opportunities for Science Journalism

by Andrew McGregor

When I was teaching English in Beijing I faced a problem of how to explain the concept of a nerd to a Chinese student.

“It’s someone who studies a lot and is good at math and science, but they do it too much.”
“Like Bill Gates?”
“Yeah.”
“And it is bad to be a nerd?”
“Yes, kind of.”
“Why?”

Without the cultural context of life in America it was very difficult to explain the concept of what a nerd is.

The internet is dragging science journalists into the future with demographically violent trembles affecting the entire field.  The internet allows unparalleled access to science news and information, but this access comes at the price of analysis and context so crucial to science journalism.

Internet search functions are predicated on connectivity and the popularity of websites: the googleization of information.  Within this process there is no human arbiter to provide the appropriate scientific context and meaning.  One can use the internet to access studies from labs across the planet on esoteric components of quantum mechanics.  However, without an appropriate scientific context it is unlikely that the findings will be adequately conveyed or understood.

Essentially, the internet is facilitating unprecedented access to information but this is coming at a direct cost of science writers being able to explain what it means.

Internet use also means that science writing will have great opportunities like the exploration of online programs that can allow a user to be a climatologists and run tests on what adding carbon dioxide will do to the atmosphere and things of this nature.

It is uncertain what science journalism will look like as the internet continues to change the way media is consumed.  What is clear is that science journalists will have to play a dual role of protecting the intellectual credibility of their craft while employing the nascent technologies that may lead to its degradation.

Verbs for Scientists

Part of the problem in communicating science to the public is the scientists themselves.

“Scientists have a responsibility to communicate what they do,” said science journalist Tom Siegfried this morning.  “Part of the whole purpose of science to communicate what you found out to the world at large.”

Michael Lemonick, who has declared himself the morning’s iconoclast, argued that that would require too much of scientists.  “You’re asking people to do something beyond their job,” he said.

Even if communicating with the public is not a responsibility of a scientist, it is an essential part of being successful.  In order to write a successful grant, you must convince the funders that your research is important.  In order to get a faculty position, you must explain your work to people in different specialties.  Even fellow scientists will have no idea what you’re talking about unless you can explain it in something close to plain English.

Scientists usually like to talk about their work, and if they have achieved any prominence, they are usually very good at talking (and writing) about it.  But we are not taught to communicate.

Even though we are funded by public money (the NIH is paying for my PhD!) and work in the public interest, communication with the public is not emphasized.  Instead, we learn how to talk in the jargon of our field, to write scientific papers, grants, and give research presentations.  We use words like elucidate, attenuate, and potentiate.  When we do talk about the significance of our research, we are instructed to condense it into one long, unreadable sentence; “These studies may help elucidate the role of the neurotransmitter transporter GAT1 in contributing to epilepsy, stroke and excitotoxicity.”

Cartoonist Larry Gonick said that there should be a class called “Verbs for Scientists.”
I say “Please!”

Seeing is Believing

Science is complicated.  Seeing it with your own two eyes is almost impossible these days, even for scientists.  Some things are too small (the spin properties of a single molecule) and some things are much too big (the mysteries of the universe), and some things are just plain invisible.  However, the closer we can get to “seeing” something, the more we believe it.  Consider how reputable the TrimSpa creators look in their white lab coats, while they stand among beakers and flasks.  However, eye-witness accounts are rarely found in science stories. 
This is why explaining science is so damn hard.  Rarely does the public catch a glimpse of a real scientist in action.  And even if they could, who knows if they’d enjoy watching?  Often, the crawl of scientific evidence doesn’t make for a compelling scene.  But then again, maybe it would. 
According to some conference participants, some websites now have “lab-cams” to catch scientists in action.  This means that someone, somewhere, wants to see what goes on behind closed laboratory doors.  As a young(ish) person starting out with the hopes of communicating science, this is an important lesson.  I don’t mean to say that watching a researcher pipette her sample into tubes is inherently beneficial.  But rather, knowing something about the set-up of a lab or the smell of a certain room lends itself to a deeper understanding.  And these days, science is in desperate need of de-mystification.

What is science?

Talk about getting back to the basics ... the first session of our gathering had nothing less than the theme of defining what science and writing about science is all about. There’ll be a comprehensive report provided later by the good folks from the Annenberg School, but this is just a quick rundown - aimed as much at testing the blog tool and providing a couple of links as anything else.

Don Kennedy, editor-in-chief of the journal Science, started out by noting the “terrible public confusion” over how science works. He ran through the main steps of the process - hypothesis, experimental testing of that hypothesis, efforts to falsify the hypothesis based on the facts, etc. He also provided a rundown of the types of research that Science finds most intriguing: work that confirms a hypothesis that’s important but hadn’t been confirmed before (say, global warming?), counterintuitive breakthroughs (say, the Chicxulub asteroid) and explorations of new territory that the editors “jump out of our shoes about.” Kennedy said the “secondary market [that is, journalists] is pretty good at picking out what matters and what doesn’t matter.”

Among the links to follow up, particularly if you want to address the ever-popular “theory vs. fact” debate, is the National Academies publication “Teaching About Evolution and the Nature of Science.”

Alex Witze, senior news and features editor at Nature, weighed in with her outsider-turned-insider view of the scientific publishing process. “Journalists in general do not recognize how sloppy science can be,” she noted. Peer review, for example, is “no inoculation against stupidity.” But there are ways to fight the madness.

She recommended getting up to speed on statistics, citing the classic work “News and Numbers.” ... and she recommended using a diversity of sources as your truth squads for scientific claims (not always turning to, say, John Pike, Keith Cowing and Art Caplan, for example).

She also put in a plug for Connotea, a social recommendation site for scientists a la Del.icio.us.

Michael Lemonick, a veteran of Time magazine, took on the “iconoclast” role: He cited the example of an astrophysicist who thought he discovered planets circling a pulsar - then, weeks before giving a big presentation, found a flaw that led him to reverse his views. He was persuaded to give the presentation anyway, and won a standing ovation for it.

Similarly, journalists should have a strong ethic of questioning what is thought to be known. “I don’t think we question ourselves enough,” he said. And in keeping with the iconoclastic point of view, Lemonick questioned whether having more science-friendly editors would solve the problem. “The proposition that ‘if only we could do it more, things would be better’ ... I think we’re kidding ourselves.”

In the discussion part of the session, we talked about the implications of a world in which embargo times were more fluid, or really hardly existed at all ... as well as some of the cautionary tales about going off half-cocked in science reporting. Some examples and links:

- The Arxiv site for physics papers, which is eroding the embargo paradigm.

- A controversial study about the origins of corn.

- Pyramids made out of concrete?

- The Bosnian pyramid.

By the way, all these links are purely plucked out by me - in some cases, recommended by presenters, and in some cases not.

Science Journalism Faces Important Challenges

by Andrew McGregor

In the film “Goodbye Lenin” the protagonist’s mother suffers a stroke just before communism falls in East Germany and he tries to maintain her perception that communism has not ended.  One of the ways he does this is by playing old news broadcasts because in communist East Germany the news never changes.

That moment was funny and interesting but it also sheds light on journalism everywhere.  If someone were to randomly be given a newspaper with the headline “SEX SCANDAL ROCKS CAPITOL HILL” they would be hard pressed to determine exactly when that headline was written.  Mark Foley, Bill Clinton? Who? When?

Science journalism is different because when a story is reported the category of science gives the article validity not possessed by other kinds of journalism.  For example, the headline “FIRST TEST TUBE BABY BORN” is an important moment of scientific progress: a headline that stands alone in history.

The session this morning very much dealt with such issues as luminaries from around the nation debated the nature of science writing, starting with such fundamental considerations as asking, “What is science?”

Dialectical opinions about what to do with science journalism were expressed, but the salient issue became the problem of science and belief.  Donald Kennedy, the editor-in-chief of Science Magazine gave an example by dropping a pencil on the floor as an example of people believing in gravity.  He contrasted this with a belief in evolution, stating that only about half of the American population believes in evolution.

This statistic belies the seminal role of science in terms of cultural and journalistic importance.  While journalism in its idealized manifestations is supposed to be fair, objective, and balanced.  What science journalism states is perceived as being true by the public.  This is part of the uneasiness faced by science writers as they must be both educators and reporters; forcing science writing outside of the traditional tropes of journalism.

In terms of journalism it is interesting to contrast science reporting with political reporting.  People expect politicians to lie on some level.  This cynicism gives itself to different styles of reporting where George Bush says A and Nancy Pelosi retorts with B and that can be a complete story.

Science journalism is held to a different standard commensurate with the way science is perceived not as another story, but as something true.

Science writers and scientists themselves will not tend to argue that their research is a unifying and complete theory of life.  However, the perception is there as scientists are expected to provide missile defense shields, cures for AIDS, and a narrative for the creation of existence.

It is this expectation that makes science journalism an intrinsically difficult and important part of contemporary culture.

Science is Messy

The question of the morning is “What is Science?”

Don Kennedy, editor of the journal Science, says “It’s a process.”

“It’s a dirty process,” clarifies Alexandra Witze, news editor at Nature, “Science is quite messy.”

As a graduate student in Neuroscience, pretending for the next few days to be a journalist, I can tell you firsthand that Witze is right.  Performing a scientific experiment often seems akin to making a soufflé without a recipe. It usually doesn’t work.  You make a hypothesis, you figure out how you might test it, and you try it.  It doesn’t work.  Not the first time, not the second time, not the third time.  You re-assess your experiment. You try it again, and again… and again. Perhaps, if you are lucky, you get a result. Perhaps it supports your hypothesis. Perhaps not. 

Someday, after several years of work, you might have enough data to publish a paper on a tiny piece of a puzzle, a small question within the world of science, which might elucidate something that has some small relation to some bigger question.  As a scientist, this is how I see science.

In journalism, it’s what’s new that matters.  An article that was published in Nature yesterday is something that scientists “just discovered.” In fact, those scientists were probably working on that discovery for years, and have known for months the conclusion that today is “news.”

Yet even when scientists have amassed enough evidence to support their hypothesis and publish in a major journal like Science or Nature, their findings still may be proven wrong. 

As Kennedy said, “Anything published in Science is ready for reversal.”

Or, as Witze put it, “You can publish in Nature or Science and it can be total crap.”

This idea of “science as process” creates a dilemma for those who need to report “science news.” They must share new findings with the general public.  They must understand the process behind it.  They must help convey the process in their writing, to fit a new finding into the context of the history of the field.  And, they must accomplish all this in 400 words. 

Science Reporting--the Messy, Warty Truth

Science is a road, not a destination.  Although the morning session was intended to define science (a challenge, to be sure), many participants stayed away from hard and fast definitions—probably because none exist.  Many alluded to the idea that science is a process, a messy, complicated, slow process.  To set the tone of the session, Donald Kennedy invoked Karl Popper’s theory of falsification.  Good science hinges on Popper’s ideas.  In other words, hypotheses can only be proven wrong, not right.  Unexpected results and setbacks mean that it’s working.
Alexandra Witze said that science is “messy and full of warts.” This dirty, complicated business is a far cry from the pristine, monumental breakthroughs that are so often reported as such in the media.  How, then, does the true nature of scientific progress impact the way in which its stories are told?  Have the readers’ expectations been modeled after traditional news subjects, where there is always a clean punchline?  By trying to compress the grueling, convoluted path to scientific results into parcels of easily accessible blurbs, are we doing a disservice to the public? 
Scientists design questions to disprove their hypotheses.  However painful it may be, the experiments that prove their ideas wrong are necessary.  The ability to doubt themselves and continually question their ideas is what drives scientific progress.  Michael Lemonick implored this group f journalists to lose some of their self-assuredness and assume the insecurities that scientists know so well.  Science journalism cannot afford to be fat and happy—things are changing, and we must adapt.  The destination, if one exists, is far away, and unlike anything people imagined.  But thinking and talking about these issues—science on the internet, the merits of embargo system, shrinking reporting space—is an important stop on the road. 

Lemonick’s challenge

What are we after, here?

We want to engender a trust of scientific evidence, through an appreciation for the process that produces that evidence.  We want people to know why they believe what they believe. Many participants emphasized (or implied) that focus on scientific process is the way to achieve this, and I believe they’re right.

However, Michael Lemonick pointed out that there has been a great deal of excellent science writing that has done exactly this. Many writers have pointed out the fallibility/reversibility of science, stressed process over facts, and highlighted the difference between common and scientific usages of terms like “theory.” Many of the attendees have written stories conveying the passion and drama of science. Even with all of these great pieces of writing, half the population still disbelieves evolution and trusts astrology.

The proposition that if we could only produce more articles about science process, things would be different - Lemonick said - is kidding ourselves. People have read these stories, and the world hasn’t changed. Maybe more stories would have more effect - but what reason do we have to believe that editors will include more stories now, hearing the same arguments they always have?

KC Cole pointed out that there is such a thing as a tipping point. Continued emphasis can have an effect on editors and on the public consciousness. New media also provide a unique opportunity to collect thorough statistics on readership. Every click on a page can be counted, in a way that every page read by newspaper or magazine subscribers could not. These statistics could prove convincing to editors (and advertisers) that there ought to be more science in the media.

There seemed to be a general consensus of the types of stories that ought to be told. The questions that remain are how and where to tell those stories, and how to convince editors to run them. And Mr. Lemonick’s challenge should be kept in mind: how do we know we’re not kidding ourselves, telling good stories in the same old ways? How do we know that what we’re doing is truly New?

Welcome to the Covering Science in Cyberspace Blog

Two dozen prominent science journalists and science communicators were invited to participate in this special conference with three goals: 1) Identify the critical issues facing science journalists in the digital age; 2) identify innovative forms of multimedia story-telling and presentation of complex issues online; and 3) identify “best practices” for coverage of science issues on digital platforms. Among the topics discussed were:

• Defining exactly what is “science”;
• Revealing untold science stories and determining why they have not been told;
• Exploring visual journalism and digital story telling techniques.

Knight Digital Media Center welcomes comments from readers related to the journalists’ specific discussions or related to the more general topic of science journalism. To post your comments, please browse the blog posts once the seminar has begun.