In September 2011, researchers at the CERN institute in Geneva convened a press conference to make a jaw-dropping announcement. The scientists had been firing a beam of neutrinos 450 miles from Switzerland to Italy, and, after taking tens of thousands of measurements, they'd discovered that some of the neutrinos appeared to be moving faster than the speed of light. If true, the results would've shattered some of the most cherished beliefs of physicists.

What was almost as surprising, however, is that the scientists had not yet published their findings in an established scientific journal at the time of the announcement. The results had not been picked over by expert reviewers who could try to poke holes and question the conclusions. Instead, the CERN researchers were placing their paper on arXiv, submitting it to an "open-source review" in which anyone from around the world could scrutinize the experiment and try to verify the results.

Six months later, the findings of the so-called OPERA experiment were disproved. Four other experiments had found that neutrinos travel at the speed of light after all. Einstein's theory of special relativity was saved. Yet the CERN researchers also set an odd precedent, in which scientific results are announced to the world and hyped before they've gone through traditional peer review. To some scientists, this was a controversial practice. And now the practice seems to be creeping to other areas, including the always-charged field of climate science.

Over the weekend, two groups of researchers noisily announced the release of new climate-science papers they'd written. Physicist Richard Muller's team at the Berkeley Earth Surface Temperature study (BEST) declared that it had re-confirmed the temperature results of NASA and other groups. The Earth has indeed been heating up, the group found. Muller even wrote a New York Times op-ed to showcase his findings. Meanwhile, climate skeptic Anthony Watts trumpeted a new paper that questioned some of the techniques used by NOAA to calculate U.S. temperature trends. Watts' paper was quickly heralded by climate-change doubters.

And yet, as my colleague Jason Samenow discusses in detail, neither of these research endeavors have yet undergone full peer review. Watts said he's planning to submit his paper to a journal, while Muller's group conceded that their studies still haven't made it through review. That makes these papers no different from thousands of others around the world waiting to be published by journals. So why should these findings receive special hype?

The idea behind peer review, after all, is that experts in the relevant fields can scrutinize the research, find potential problems, and offer corrections if need be. Watts's paper, for example, contests the methods that NOAA uses to analyze its temperature data. His criticisms might turn out to be useful. Or they might be off-base. But a non-expert would be hard-pressed to figure that out. (For those curious, Victor Venema, a scientist who does work in a related field, took an early look at Watts' paper and offered some words of caution.)

One possibility is that these papers are so crucial that they can't possibly wait years before being vetted. That was one rationale behind announcing the faster-than-light neutrino result. Einstein might be wrong! That's big news. But is that true in the case of these climate papers? Elizabeth Muller, the daughter of Richard Muller and a co-founder of the BEST project, tried to suggest as much: "I believe the findings in our papers are too important to wait for the year or longer that it could take to complete the journal review process."

And yet, as Samenow argues, this defense is hard to justify. The BEST research doesn't revolutionize climate science. The research, if it holds up, basically confirms, and possibly improves a bit, the existing temperature records that show the earth is warming. As Penn State's Michael Mann quips, Muller's results have "brought him up to date [with] where the scientific community was in the the 1980s." Granted, Muller's results may garner some political interest, if only because he has criticized climate scientists in the past and because his study received $150,000 from the conservative Koch Foundation. But is that reason enough to race ahead of the review process?

Meanwhile, Watts's research may well prove useful for helping NOAA improve its methods for estimating the increase in U.S. temperature in the United States since the 1970s. (Though, as Samenow notes, there's a fair bit of peer-reviewed research backing NOAA's temperature records, so it's worth holding off judgment on that.) But there's little chance that Watts's paper will upend the larger picture on climate change. The United States makes up just 2 percent of the Earth's surface. An adjustment to the U.S. record would have, as noted climate skeptic Luboš Motl concedes, a "negligible" impact on global trends. This isn't quite in the same category as overturning Einstein.

Critics of the traditional peer-review process contend that it can be clumsy, slow, and limited to just a handful of expert reviewers who may be biased and are unaccountable for their judgments. Wouldn't it be better, they ask, to harness the vast collective mind of the Internet to dissect research? Here's Elizabeth Muller again: "Our papers have received scrutiny by dozens of top scientists, not just the two or three that typically are called upon by journalists." And Watts, for his part, says that he's circulating a pre-publication draft paper because Muller did it first.

Yet many climatologists have countered that there's no good substitute for letting peer review do its work, especially in a fraught field like climate science that attracts lots of public attention. A few years ago, Penn State's Michael Mann and NASA's Gavin Schmidt penned a defense of the peer-review process for Real Climate. Yes, they noted, bad papers do get past reviewers. Just because something is published in a journal doesn't mean it should be taken as gospel. But the current system works remarkably well:
Put simply, peer review is supposed to weed out poor science. ... [E]ven when it initially breaks down, the process of peer-review does usually work in the end. But sometimes it can take a while. Observers would thus be well advised to be extremely skeptical of any claims in the media or elsewhere of some new "bombshell" or "revolution" that has not yet been fully vetted by the scientific community.
On a related note, Mann and Schmidt also offered a word of caution for those - especially in the press - who get the urge to overplay any single new piece of climate research:
The current thinking of scientists on climate change is based on thousands of studies (Google Scholar gives 19,000 scientific articles for the full search phrase "global climate change"). Any new study will be one small grain of evidence that adds to this big pile, and it will shift the thinking of scientists slightly. Science proceeds like this in a slow, incremental way. It is extremely unlikely that any new study will immediately overthrow all the past knowledge. ...

Yet, one often gets the impression that scientific progress consists of a series of revolutions where scientists discard all their past thinking each time a new result gets published. This is often because only a small handful of high-profile studies in a given field are known by the wider public and media, and thus unrealistic weight is attached to those studies. New results are often over-emphasised (sometimes by the authors, sometimes by lobby groups) to make them sound important enough to have news value. Thus "bombshells" usually end up being duds.
A fair warning.