Notes on Global Warming and Abuse in the Peer Review Process

Source: Informath

by Douglas Keenan


Following discusses one aspect of how the peer review process affects the study of global warming.

The problems with the peer review process have implications for our understanding of global warming (as well as for science generally). Once something has been published in a peer-reviewed journal—particularly a prestigious journal—it tends to be considered as established, possibly even heralded as “truth”. This means that other researchers will often rely on its conclusions, with little, if any, further checking. The extent to which this happens varies among different branches of science. It seems to be especially so in the study of global warming.

The primary body tasked with advising government policy makers about global warming is the IPCC (Intergovernmental Panel on Climate Change). Policy makers generally regard the IPCC as authoritative. The IPCC bases its analyses on peer-reviewed research, but it does no checking of that research itself. Yet most peer-reviewed research is not properly checked prior to its publication. In other words, most of the research that is relied upon by the IPCC, and thus government policy makers, has never been properly checked. That probably seems incredible; it is unfortunately true.

See also

Remarks on Keenan [Energy & Env., 2007]
Statistical errors in the “hockey stick” graph


Following are some remarks about my report “Grape harvest dates are poor indicators of summer warmth”, as well as about scientific publication generally.

On 18 November 2004, Isabelle Chuine and co-workers published a research paper on global warming. The paper appeared in Nature, the world’s most highly-regarded scientific journal. And it gathered some publicity. Chuine et al. claimed to have developed a method for estimating the summer temperature in Burgundy, France, in any given year back to 1370 (based on the harvest dates of grapes). Using their method, the authors asserted that the summer of 2003 was by far the warmest summer since 1370, in Burgundy.

I had been following global warming studies only as a disinterested outside spectator (and only occasionally). Someone sent me the paper of Chuine et al., though, and wondered what I thought of it from a mathematical perspective. So I had a look.

To study the paper properly, I needed to have the authors’ data. So I e-mailed Dr. Chuine, asking for this. The authors, though, were very reluctant to let me have the data. It took me eight months, tens of e-mails exchanged with the authors, and two formal complaints to Nature, to get the data. (Some data was purchased from Météo France.) It is obviously inappropriate that such a large effort was necessary.

Looking at the data made it manifest that there are serious problems with the work of Chuine et al. In particular, the authors’ estimate for the summer temperature of 2003 was higher than the actual temperature by 2.4 °C (about 4.3 °F). This is the primary reason that 2003 seemed, according to the authors, to be so tremendously warm.

There is also another reason. The three warmest years on record, prior to 2003, were 1945, 1947, and 1952. (The instrumental record goes back to 1922, or even 1883 if we accept some inaccuracies.) The estimate of Chuine et al. for the summer temperature in each of those years was much lower than the actual temperature.

That is, the authors had developed a method that gave a falsely-high estimate of temperature in 2003 and falsely-low estimates of temperatures in other very warm years. They then used those false estimates to proclaim that 2003 was much hotter than other years.

The above is easy enough to understand. It does not even require any specialist scientific training. So how could the peer reviewers of the paper not have seen it? (Peer reviewers are the scientists who check a paper prior to its publication.) I asked Dr. Chuine what data was sent to Nature, when the paper was submitted to the journal. Dr. Chuine replied, “We never sent data to Nature”.

I have since published a short note that details the above problem (reference below). There are several other problems with the paper of Chuine et al. as well. I have written a brief survey of those (for people with an undergraduate-level background in science). As described in that survey, problems would be obvious to anyone with an appropriate scientific background, even without the data. In other words, the peer reviewers could not have had appropriate background.

What is important here is not the truth or falsity of the assertion of Chuine et al. about Burgundy temperatures. Rather, what is important is that a paper on what is arguably the world’s most important scientific topic (global warming) was published in the world’s most prestigious scientific journal with essentially no checking of the work prior to publication.

Moreover—and crucially—this lack of checking is not the result of some fluke failures in the publication process. Rather, it is common for researchers to submit papers without supporting data, and it is frequent that peer reviewers do not have the requisite mathematical or statistical skills needed to check the work (medical sciences largely excepted). In other words, the publication of the work of Chuine et al. was due to systemic problems in the scientific publication process.

The systemic nature of the problems indicates that there might be many other scientific papers that, like the paper of Chuine et al., were inappropriately published. Indeed, that is true and I could list numerous examples. The only thing really unusual about the paper of Chuine et al. is that the main problem with it is understandable for people without specialist scientific training. Actually, that is why I decided to publish about it. In many cases of incorrect research, the authors will try to hide behind an obfuscating smokescreen of complexity and sophistry. That is not very feasible for Chuine et al. (although the authors did try).

Finally, it is worth noting that Chuine et al. had the data; so they must have known that their conclusions were unfounded. In other words, there is prima facie evidence of scientific fraud. What will happen to the researchers as a result of this? Probably nothing. That is another systemic problem with the scientific publication process.

Chuine I., Yiou P., Viovy N., Seguin B., Daux V., Le Roy Ladurie E. (2004), “Grape ripening as a past climate indicator”, Nature, 432: 289–290. doi: 10.1038/432289a.

Keenan D.J. (2007), “Grape harvest dates are poor indicators of summer warmth”, Theoretical and Applied Climatology, 87: 255–256. doi: 10.1007/s00704-006-0197-9.

Instrumental temperature data for Dijon (Burgundy) is available from

Model-estimated temperature data is available from

Douglas J. Keenan