In April, I went to the Science Literacy Workshop in Berkeley, Calif. Here are a few things I took away from it — which have been useful for my newsroom as some reporters try to slog through scientific research.
1) Be wary of press releases from universities or any group that touts new research.
Before we jump in we should ask ourselves: Who’s funding this research? You’d be surprised how often interest groups sponsor university research. Has it been peer reviewed? Is the journal respected?
2) Check for sample size.
Even peer-reviewed journal articles have flaws. Read the abstract first, and then read the whole study. Check for sample size. Is the author claiming a huge scientific breakthrough from data taken out of a tiny sample?
3) Reporting on current studies is a minefield.
A stage two study, for instance, has established very little in the way of fact. The researchers have just started gathering data for publication, which means the study is not-peer reviewed and any claims from the study are preliminary. Think of it this way, the more publicity a researcher can get about his project in the early stages, the more money he’s likely to raise.
4) Call another scientist familiar with the topic.
When in doubt, call another scientist familiar with the topic, but not involved in the research. Ask her to assess the validity of the work. Also, ask the scientist who authored the report about what the study DOESN’T tell us. Dig deep into the limitations of the study. These people are grilled by their peers in the publishing process (hopefully) and any honest scientist will be up front with pointed questions. If they’re defensive or downplay the study’s shortcomings, it’s a red flag.
