The Blackboard

At Climate Audit, Bender periodically wonders whether anyone knows the precision of a single run of a Global Climate Models (GCMs). My answer is: ±0.08K. Sort of. That is, for Hansen’s Scenarios A, B & C as they apply to the decade from 1973-1983. I’ll explain what I mean by precision, why I picked that batch of years to estimate the precision uncertainty.

GISS II: Ran Three Cases with Identical Forcings

Hansen et al. 1988 described results from the GISS Model II simulations of the earth’s climate under three scenarios. Model runs for all three scenarios were initiated using the identical conditions thought to describe an equilibrium climate that would exist if atmospheric conditions matched those of 1958 for a long time. The “clock” was then begun, and forcing associated with increasing levels of Green House Gases (GHG) were varied from year to year. However, all three scenarios used identical forcing forcing from 1958-1983.

That is to say: With regard to the period from 1958-1983, the only reason model predictions differ is precision uncertainty inherent to the GISS model itself. This precision uncertainty does not account for variations in parameter values: It is due to internal functioning of the model alone.

So, if the GISS model is absolutely precise, all three scenarios should provide identical predictions from 1958-1984. The extent to which the differ is the lack of precision. Here is a graph of model results:

Figure 1: GISS temperature anamolie values are taken from Real Climate and most specifically temperature anamolies. GISS Met values are unadjusted.

Calculation of the Precision uncertainty

To calculate the precision uncertainty in the model, I did as follows:

1. Tabulated the temperature anomolies for Scenarios A, B and C for each year. That gave 3 values of “T” per year.
2. Calculated the sample standard devation ased on three data points for each year using Excels stdev() function.
3. Calculated the average standard deviation for the final 10 years when the forcing were held constant. That is, I determined the average of the standard deviation from 1974-1983 inclusive. This value is σ=0.08K.

Calculating and reporting this sort of precision uncertainty is routine when physical experiments are performed. (It’s also fairly common when modelers report results of Direct Numerical Simulations, as the precision uncertainty of an experiment is of great importance to those who wish to compare experimental results to theories or just to understand physical phenomena.)

Why use only 1974-1983?

The reason I limited calculation to later years is simple: I’m pretty sure the computations forced all scenarios to have identical temperature anomolies on the model equivalent midnight, Dec. 31 1958. For this reason, early predictions will appear to agree with each other simply because the model has not run long enough for any imprecision to manifest itself. So, for example, during the first 10 year, the 1-σ precision uncertainty is only σ=0.03K rather than the σ=0.08K I get based on the final 10 years.

The evolution of the 1-&sigma precision uncertainty with time from the initiation of experiments is shown below:

So, is 0.08K an upper or lower bound on the precision uncertainty?

This calculated precision would represent an absolute lower bound. Factors that would increase it are:

1. The 1-σ precision uncertainty may increase or remain constant as computational time increases.
2. The 1-σ precision uncertainty would increase if simulations were run with slightly different initial conditions.
3. The 1-σ precision might increase if there is any uncertainty in the magnitude of parameters in sub-grid models. (For example, modifying a parameter in a transport model for heat in the ocean over some realistic range might affect predictions. The same holds for modifying the value of any parameter.

How much larger than 0.08K could the GCM precision uncertainty be?

I don’t know.

Presumably, climate modelers have done sensitivity studies varying the magnitude of various parameters in sub-grid models. If so, they should be easily able to provide quantitative estimates of the sensitivity based on model computations they have already performed. If few sensitivity studies have been performed, the estimates might be crude, but presumably these estimates exist.

In fact, it would be highly unlikely sensitivity analysis of some sort have never been done. After all, proving a GCM model’s prediction are sensitive to the magnitude of a particular parameter is generally required to motivate detailed research to improve the sub-model for a particular physical process involved in climate.

If the model predictions are insensitive to variation in a parameter value in a submodel or to using different formulations for the submodel, there is no reason to fund research to improve the submodels.

Research to improve these models is funded, and published; so one might suspect varying the values of these parameters might well affect GCM model predictions.

What’s the precision uncertainty these days?

Of course, the values I have provided are crude estimates, and in any case, apply only to the GISS II model. Models have evolved, and one might ask the current level of precision for individual runs, or better, for ensembles of runs that might now be used to extrapolate into the future.

Unfortunately, I have no idea what that answer might be. In his post at Real Climate, Gavin Schmidt states:

Nowadays we would use an ensemble of runs with slightly perturbed initial conditions (usually a different ocean state) in order to average over ‘weather noise’ and extract the ‘forced’ signal. In the absence of an ensemble, this forced signal will be clearest in the long term trend.

If ensemble runs are available, modelers would, presumably, routinely calculate and report both the mean and a 1-σ precision uncertainty for predictions from any set of ensemble runs. It is rather standard to report both so, presumably, the information exists.

I’ve only just begun looking at climate change data, so, I haven’t looked for this information. Not having looked, I naturally haven’t found it. If anyone knows where it might be, I’d sure be interested.

In the meantime, the only information I’ve run across tells me: The precision uncertainty of one single run is no better than 0.08K when predicting values roughly 20 years in the future.