I am 95% shocked

You may remember the public announcements of the discovery of Higgs boson, which prominently featured the concept of statistical certainty used by the physicists:


The percentage figure quoted in the linked article has been derived from billions of observations hinting at the existence of the mysterious particle. The physicists can be 99.999999999% certain that Higgs has been found because the results of these observations have been processed using a statistical procedure which produced this particular percentage figure. If another lot of physicists looked at the same results their certainty would be exactly the same – there is no judgement involved here, only pure maths.

As an engineer I can relate to it because certainty has the same meaning when applied to the properties of building materials. In design we typically use the “95% confidence” strength of steel or concrete. This is the figure which we expect to be exceeded by 95% of samples, assuming the normal statistical distribution based on the test results. If a different engineer looked at the same set of concrete strength test results, his “95% confidence” numerical value would be the same as mine – there is no scope for subjectivity in the process.

In this context I was quite intrigued when the IPCC 5th Assessment Report recently announced that:

It is extremely likely [which in IPCC-speak means 95% confidence] that more than half of the observed increase in global average surface temperature from 1951 to 2010 was caused by the anthropogenic increase in greenhouse gas concentrations and other anthropogenic forcings together.

The 4th AR issued in 2007 ascribed only 90% confidence to a similar statement so the certainty of the attribution has increased in the last few years. Having some background in applied sciences I accepted that the results of the observations, morphed into a normal statistical distribution, are now pointing to a 95% certainty. They have done the maths, right?

Well, not exactly. The linked article from Dr Judith Curry’s blog explains that the IPCC certainty is not directly based on any maths at all:


An excerpt explaining how the IPCC have arrived at the widely quoted 95% certainty on the human attribution of most of the observed global warming since 1951:

Yesterday, a reporter asked me how the IPCC came up with the 95% number.  Here is the exchange that I had with him:

‘Reporter:  I’m hoping you can answer a question about the upcoming IPCC report. When the report states that scientists are “95 percent certain” that human activities are largely to cause for global warming, what does that mean? How is 95 percent calculated? What is the basis for it? And if the certainty rate has risen from 90 n 2007 to 95 percent now, does that mean that the likelihood of something is greater? Or that scientists are just more certain? And is there a difference?

JC:  The 95% is basically expert judgment, it is a negotiated figure among the authors.  The increase from 90 to 95% means that they are more certain.  How they can justify this is beyond me.

Reporter:  You mean they sit around and say, “How certain are you?” ”Oh, I feel about 95 percent certain. Michael over there at Penn State feels a little more certain. And Judy at Georgia Tech feels a little less. So, yeah, overall I’d say we’re about 95 percent certain.”  Please tell me it’s more rigorous than that.

JC:  Well I wasn’t in the room, but last report they said 90%, and perhaps they felt it was appropriate or politic that they show progress and up it to 95%.

Reporter:  So it really is as subjective as that?

JC:  As far as I know, this is what goes on.  All this has never been documented.’

I am finding it absolutely astounding. The 95% certainty claimed by IPCC is an “expert opinion” of the selected few involved in preparing the summary of the report and is not based on the same statistical procedure physicists and engineers adhere to. If different people were in the room their certainty level might have been different. Moreover, we do not even know how certain each particular scientist helping write the summary was and how the individual numbers were aggregated to produce the final certainty put in the report.

I am 95% certain that 95% of people reading the IPCC report do not realise that this is how the 95% certainty level has been arrived at.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: