The quality of thinking inScientific American has not been impressive lately. In that publication articles with lazy logic and weak arguments aren’t simply published, they’re published twice. This paper is a great example.
Titled The Physical Science Behind Climate Change, and written by five Intergovernmental Panel on Climate Change (IPCC) insiders, it appeared firstin July 2007. Perhaps this was the magazine’s way of demonstrating support for the new edition of the IPCC’s climate bible released that same year.
The second time, in October 2008, the article was part of a package celebrating Nobel winners who’ve “written more than 200 articles for Scientific American.” When a science magazine suggests an equivalency between Peace Prize winners on the one hand and physics Nobel winners on the other, that’s not a good sign.
Nevertheless, I believe in giving credit where it’s due, and Scientific American currently has an opinion piece in its health section enumerating the shortcomings of peer-reviewed scientific research. Here are some particularly relevant quotes [bold added by me]:
Many studies that claim some drug or treatment is beneficial have turned out not to be true.Even when effects are genuine, their true magnitude is often smaller than originally claimed.
scientists are tempted to show that they know more than they do.
Much research is conducted for reasons other than the pursuit of truth. Conflicts of interest abound, and they influence outcomes.
First, we must routinely demand robust and extensive external validation – in the form of additional studies – for any report that claims to have found something new. Many fields pay little attention to the need for replication or do it sparingly and haphazardly.
At the moment.outsiders frequently do not have access to what they need to replicate studies. Journals and funding agencies should strongly encourage full public availability of all data and analytical methods for each published paper.
The overall message is that scientists are, indeed, only human. They are as vulnerable as the rest of us to bouts of arrogance and self-aggrandizement. They sometimes cut corners in their professional lives. They don’t always follow the rules. Their motivations are sometimes less-than-pure.
This means that far stricter accountability mechanisms need to put in place. Until a researcher’s data is entirely open to examination – and until his or her findings have been replicated by disinterested third parties – it’s foolish to place too much trust in those findings.
From what I can tell, if such a regime were actually adopted, well over half of the peer-reviewed papers cited by the IPCC would be immediately disqualified.
Read the whole Scientific American piece here. [backup links: page 1, page 2]
For more on the convoluted logic of the five-IPCC-author piece, see the end of this post.
