The Blackboard:

Recently, I have been interested in the answer to these questions:

How does the IPCC really make projections? When were tuning parameters in models used to create projections in the TAR “frozen”?

Both questions were motivated by one of the many of the puzzling features of a recent paper by “The Rahmstorf Six Seven”, often referred to as “Rahmstorf et al 2007.”

That paper suggested that the projections in the TAR were based on physical models which are independent of observations since in 1990. But, mysteriously, these projections were only published in 2001. The question is: is this entirely accurate? And even if it is, can we be sure of tuning parameters were not influence by temperature measurements collected after 1990?

By my reckoning, the models actually used to create the projections published in the TAR are tuned to something, and those tuning parameters were frozen sometime around 2000-2001, and no earlier. It is highly unlikely the predictions using these tuning parameters were not compared to recent data; it is quite likely that tuning parameters that resulted in poor predictions would have resulted in efforts on the part of scientists to develop better methods to select tuning parameters.

So, validation of these models should be based on data obtained no earlier than 2001; to do otherwise would result in a false validation, as the “validation” will rely on data that likely affected the choice of the tuning parameters.

What type of projections am I talking about?

The more specific question I am asking is this: How did the IPCC create projections for Global Mean Surface temperature such as those in Figure 9.13b from 9.3.3 Range of Temperature Response to SRES Emission Scenarios in the TAR:

The narrative alongside this figure explain how this figure describing the TAR projections was created:

Figure 9.13b shows the simple climate model simulations representing AOGCM-calibrated global mean temperature change results for the six illustrative SRES scenarios and for the full SRES scenario envelopes. The individual scenario time-series and inner envelope (darker shading) are the average results obtained from simulating the results of seven AOGCMs, denoted “ensemble”. The average of the effective climate sensitivity of these AOGCMs is 2.8°C (see Appendix 9.1). The range of global mean temperature change from 1990 to 2100 given by the six illustrative scenarios for the ensemble is 2.0 to 4.5°C (see Figure 9.14). The range for the six illustrative scenarios encompassing the results calibrated to the DOE PCM and GFDL_R15_a AOGCM parameter settings is 1.4 to 5.6°C. These two AOGCMs have effective climate sensitivities of 1.7 and 4.2°C, respectively (see Table 9.1). The range for these two parameter settings for the full set of SRES scenarios is 1.4 to 5.8°C. Note that this is not the extreme range of possibilities, for two reasons. First, forcing uncertainties have not been considered. Second, some AOGCMs have effective climate sensitivities outside the range considered (see Table 9.1). For example, inclusion of the simple model’s representation of the CCSR/NIES2 AOGCM would increase the high end of the range by several degrees C.

(Italics mine.)

When I first skimmed this text, I thought the these projections represented the ensemble average of actual AOGCM runs. However, on further reading, realized that is not the case. For if this were the case, there would be no reference to “two parameter settings”, nor would we see discussion of “simple climate model” that are “AOGCM-calibrated”.

The modifiers in that paragraph imply that the projections in figure 9.13b in the TAR are the product of a simple, two parameter model. “Parameters” are often referred to as “tuning knobs”; in this case, the “simple models” are “tuned” to predict what a particular AOGCM might be expected to predict if that particular AOGCM were actually run using a particular set of SRES forcings during a particular period in time.

So, these projections are the output of a model (the simple model) that predicts the output of another model (an AOGCM) that is though to simulate the climate of the earth.

Predictions are the result of a “cascade” of models.

The process for making projections is described in Section 9.3 and Appendix 9.1 of “Climate Change 2001: Working Group 1: The Scientific Basis”.

As far as I can tell, the projections in the TAR are based on what I might call a “cascade” of models, but which the TAR calls a 8.3 Model Hierarchy

I’ll describe each type of model below.

1. A number of AOGCM’s were used to simulate the earth’s climate response under a variety of forcings SRES.

   AOGCM’s the models one often reads about in various blog-climate-war debates. These are detailed models that attempt to simulate the earth by solving the equations governing conservation of mass, momentum and energy. They include parameterizations for various processes; we often here that difficulties associated with modeling clouds are thought to introduce uncertainty in the model predictoins.

   Of course, the predictions of each AOGCM differs from other AOGCM. However, since these are based on approximate models for individual physical processes that happen on earth (rather than say, astrology), one might hope that, on average, ensemble of AOGCM results predicts the earth’s climate.

   In particular, one might hope the ensemble average results of a collection of AOGCM’s predicts measurable metrics like “global mean surface temperature” (GMST) or sea level rise accurate.

   These are used only indirectly to create projections as in figure 9.13 in the TAR.

2. Simpler “upwelling diffusion-energy balance models (UD/EB)” have been developed, and are tuned to each AOGCM. ‘

   The term “upwelling diffusion-energy balance models (UD/EB)” is describing a type of simple model.

   The goal of the simpler model is to predict the results of a more complex AOGCM model. So, when run, each of these models should “predict” what a particular AOGCM) would predicted if it were run for a particular scenario.

   As far as I can determine, the simple “upwelling diffusion-energy balance models (UD/EB) models” used in the TAR contain six adjustable (aka “tuning”) parameters. (See table 9A.1. )

   Like “Lumpy”, my toy model, which contains two “tuning” knobs, the tuning knobs in the UD/EB model do relate to some physical process. The tuning knobs in the UD/EB include a climate sensitivity, diffusivity in the ocean and what not. Nevertheless, like “Lumpy”, the magnitudes are obtained by fitting to a data set created by an AOGCM.

   I infer that an “upwelling diffusion-energy balance model (UD/EB)” and its six specific tuning parameters, represents “one” model. In the IPCC process there seems to be a one-to-one relationship between AOGCM’s and “UD/EB” (aka ’simple’) models.

3. The simpler “upwelling diffusion-energy balance model (UD/EB)” are then used to predict the earth’s climate under a wide variety of forcings as specified by the SRES.

   Each of the ’simpler’ models can be driven by a SRES forcing projection. When this is done, the result is a some sort of time series for climate. Some metrics of interest, like “Global Mean Surface Temperature” can be computed from the output.

   These are the averaged and used to create plots like Figure 9.13b, which can later be compared to real earth data.

So we see that the projections are influenced by AOGCM output, but indirectly. More specifically, the predictions are those of simpler models that predict what an AOGCM would have predicted if the AOGCM has been run using a particular SRES forcing.

So, when were the “simple models” created?

If we define a “simple model” as the general class of model and its specific tuning parameters, then specific models used in the TAR appear to have been created shortly before 2001.

In Appendix 9.1: Tuning of a Simple Climate Model to AOGCM Results, we learn, “The tuning is based on the CMIP2 data analysis of Raper et al. (2001b). Note that there is often a year or two lag between final results.

As the tuning parameters were not selected until shortly before publication of Raper et al. (2001b), beginning comparisons in 2001, seems appropriate. (Susan Raper’s publication list is here.)

So, while Rahmstorf et al’s claims model predictions in the TAR are somehow independent from data since 1990, and that, in some sense, the predictions cannot be influenced by real temperature data measured after 1990, I think otherwise.

Clearly, analytical choices are made when selecting the magnitude of all six tuning coefficients to simulate the output of AOGCMs. Like it or not, even simple models tuned to AOGCM’s can be compared to actual empirical data prior to finalizing the choice of tuning coefficients.

In fact, I would amazed to discover that Raper et al. cloistered themselves away from real data after 1990. Such a idea is not only implausible, it would be absurd.

It is more likely, that out of true, honest, scientific interest, scientists made data comparisons during the process of selecting the magnitude of the six tuning parameters. Though a procedure exists to select models, that procedure itself was being developed as the coefficients were selected. Like it or not, it would be very difficult for researchers to proceed with tuning constants that resulted in poor fits to the most recent measurements of global means surface temperature.

My conclusion

The models used to create TAR projections contain tuning coefficients developed shortly before 2001. The full process for selecting the tuning constants is, itself, an ongoing research project; the method is constantly being improved. It is inevitable that the process developing the method to select the constants, and the choice of the constants themselves was influenced by the most recent data available. This would include nearly all the Global Mean Surface Temperature data between 1990 and 2000.

For this reason, the fidelity of the tuned models should be tested against data that arrived after the final selection of tuning coefficients. In the case of the TAR, this means validation should be restricted to observations no earlier than 2001.

Sections of TAR describing the process

For readers convenience, here are links to the most pertinent portions of the TAR:Executive Summary of 9. Projections of Future Climate Change.

Previous Post:
« Comparing IPCC Projections to Individual Measurement Systems.

Next Post:
Erhmm… it’s the IPCC AR4 projections that are falsified! :) »