Last year a committee struck by the InterAcademy Council (a collection of international science bodies) investigated the Intergovernmental Panel on Climate Change (IPCC). In the process, it distributed a questionnaire. Recently 678-pages of answers were released. I’ve been wading through them. [3 mb PDF here]
These 232 (mostly) anonymous individuals were asked for their views regarding how the IPCC identifies and fixes errors – including errors discovered after IPCC reports have been published. This is a crucial matter.
On January 20, 2010, the IPCC issued an embarrassing public retraction. Its 2007 Nobel-winning report had incorrectly stated that Himalayan glaciers would vanish by 2035. Several people had brought this mistake to the attention of the IPCC, but had been ignored. According to the Guardian newspaper:
Georg Kaser, an expert in tropical glaciology at the University of Innsbruck in Austria and a lead author for the IPCC, said he had warned that the 2035 prediction was clearly wrong in 2006, months before the report was published. “This [date] is not just a little bit wrong, but far out of any order of magnitude,” he said.
A two-page article published in Science on November 13, 2009 quoted another glacier expert saying the IPCC had gotten the date “horribly wrong.” The article’s author, Pallava Bagla, says he himself raised the matter with IPCC chairman Rajendra Pachauri several times earlier that month – verbally as well as in e-mails. (See here for yet another account of attempts to set the record straight.)
Individuals who answered the InterAcademy’s question about IPCC errors provided responses that are all over the map. One person intemperately describes the IPCC’s handling of these matters as “criminal” (p. 616). At the other extreme, some people don’t see any reason for concern. Apparently, the world has unfolded exactly as it should:
The responses were entirely appropriate. IPCC strongly defended the science, admitted where there was a small error and reaffirmed that this did not change the balance of the evidence. (p. 287)
Absolutely fine, with human foibles [errors are] simply not entirely avoidable. Good gracious. If every other area of science were scrutinized this carefully – we’d find a lot more mistakes and misinterpretations. It’s all overblown because of the nature of the politics of climate change. (p. 414)
I’ve been impressed. To me, the point isn’t whether every last detail in the Assessment is absolutely correct, but what the whole of the report tells us. (p. 649)
I feel that the IPCC response to the “glaciergate” commotion was entirely appropriate. They quickly went out with a press statement admitting the error and retracting the erroneous material. (p. 485)
In contrast to the above, numerous people – including many who are solidly supportive of the IPCC and its work – think there’s room for improvement. The glacier scandal was the IPCC’s first real experience of negative publicity. It’s clear some important lessons were learned as a result. On page 482 a respondent remarks:
In my opinion, the glacier error could only blow up the way it did because the IPCC was much too reluctant and took way too much time to admit it.
Adds someone else, on page 589:
What we learned in the last several months is that the IPCC was ill prepared to acknowledge and correct errors. I stumbled across the silly 2035 Himalayan glacier melt statement a couple of years ago. I sent email to a glaciologist colleague to ask how anything like this could have been written, noting that it certainly wasn’t supported by anything in the published literature. [IPCC expert reviewers] had actually caught this error, and the authors and the review editors screwed up by not correcting it.
In the view of the individual whose comments appear on page 113, it’s clear that:
The IPCC lacks a system of swiftly responding to errors in the report. There should be protocols and adequate response times when such issues come to the surface.
The person whose thoughts appear on page 3 agrees:
A procedure [regarding] how to cope with such errors is obviously lacking and should urgently be installed.
Adds someone else, on page 5:
This is not rocket science, so to speak. The fact that the IPCC Chair let the issue of small errors.fester without taking any direct action is inexcusable since it would have been so easy to address. This should be one of the explicit jobs of the IPCC Chair.
On page 95, the larger question of errors is framed as an ethical one:
Whenever an error has been discovered it is important to correct it the moment [it is] discovered. This is one of the ethics of science and scientists.
Elsewhere, the matter is viewed strategically:
Errors should be admitted humbly and to create a “bunker mentality” in defending the IPCC is the wrong strategy to follow. (p. 104)
I think the IPCC should do as all good PR companies advise, which is to be fully honest and open as soon as an error is detected, and make no attempt to cover up. (p. 443)
If any errors are discovered then the IPCC must publicly acknowledge the errors.this report is, after all, used to direct billions and even trillions of dollars.so accuracy and openness is critical. (p. 475)
Other people are concerned about the IPCC’s overall reputation:
Acknowledgment of errors and immediate correction is important for the credibility of the IPCC. (p. 258)
The mistakes made in this regard profoundly undermined the credibility of IPCC and harmed the community and the cause of science. The damage control was rather damage amplification. (p. 394)
There were long delays in responses to criticisms in the press.The IPCC has no press response strategy in place and is unequipped to deal with crises. It has generally been poor in its attempts to retain public confidence. (p. 657)
Adds someone else, on page 294:
Errors discovered after publication have never been dealt with by the IPCC, which is a major deficiency. All authors are probably aware of minor errors in their chapters, but there has hitherto been no mechanism for correcting known errors, let alone possible errors that emerge in the public domain, post assessment.
The person on page 386 suggests that an independent auditor should perhaps determine whether or not an error has occurred:
Possible errors discovered after publication should be handled by some kind of auditor, sooner rather than later, and IPCC leadership should refrain from commenting before the auditor has announced his findings.
Despite the large amount of intelligent commentary in this document, there’s no shortage of odd remarks, as well. One person says:
I’d like to see IPCC have a process for dealing with claims of problems; what I fear, however, is that if IPCC does have a process, it will get abused by the critics claiming all sorts of things such that an IPCC assessment will never be considered done. [p. 314, bold added]
Should society decline to set up checks and balances (say, regarding police conduct), because there’s a chance some people will abuse the complaints process? Is sorting valid complaints from frivolous ones not an integral part of many human endeavours?
In another vein, IPCC reports are surely written because we need reliable information. It’s therefore bizarre to see someone on page 485 attempting to minimize the glacier error by saying:
the fact remains that glaciers in the Himalayas ARE melting – just not this quickly.
Surely the rate at which the melting is occurring is the important point. If the matter isn’t going to be a concern for several millennia why would we even be talking about it?
That some people who fill senior positions with the IPCC see the world in highly politicized – as opposed to soberly scientific – terms is demonstrated by the Coordinating Lead Author whose remarks appear on page 412:
In a report of 1000 pages it is impossible not to have a few small errors – as mistakes made in good faith. Newspapers do this all the time and their errors are accepted and forgotten. IPCC is subject to a deliberate campaign to discredit its work and must be much less defensive in its responses. Vested interests have tried in the past to discredit and postpone the implementation of policies in relation to cancer and smoking, ozone depletion, nuclear winter and now climate change.
Remember, this answer was provided in response to a question about how the IPCC deals with errors. How can smoking be remotely relevant here? What does it say about the judgment of the IPCC that a person who indulges in such partisan rhetoric has been responsible for overseeing a report’s chapter?
Readers of this blog know I am often critical of the media. But it should be pointed out that newspapers traditionally provide the “first draft of history.” News accounts are often written in a hurry, as the story is developing, before anything like all the information is available. It strikes me as bizarre that errors by an organization that claims to produce a gold-standard, rigorous high quality scientific report would be compared to those in a daily newspaper.
Among this vast array of views, perhaps the most provocative originates with another senior IPCC official. This person has been a Bureau Member (a select, administrative group currently comprising 31 individuals). In the opinion view of this individual, outside scrutiny of the IPCC’s report is not welcome. Indeed, it should be discouraged.
Members (governments) of the IPCC and observers should be encouraged to participate as actively as possible during the IPCC review process, to detect errors as much as possible within the assessment cycle. Unilateral or pluri-lateral initiatives by.observer organizations to carry out additional review processes after the publication of the reports should be discouraged. [p. 212, bold added]
Well here’s the problem. Members of the public are being told that their lifestyles have to change radically because of what the IPCC report says. They’re being told that their national economies must be re-engineered from top to bottom because of the IPCC report. Moreover, the public is expected to fund the profound changes the IPCC report is being used to justify.
No reasonable person should expect any of the above to occur in the absence of intense and enduring public scrutiny.
