Do any of you remember Dominguez et al 2008, a paper that discussed ocean heat content revisions? One of the things the paper showed was that after correcting the observations of ocean heat content, AOGCM projections that include volcanic eruptions better match observations of ocean heat content.
This needless to say, the long-standing mis-match vexed the modeling community. Not surprisingly, the paper resolving the mis-matched gladden the heart of Gavin, who wrote a post entitled “Ocean Heat Content”.
Discussing the mismatch, he said:
But remember that the second big issue with ocean heat content trends is that they largely reflect the planetary radiative imbalance. This imbalance is also diagnosed in climate models and therefore the comparison serves as an independent check on their overall consistency. Domingues et al show some comparisons with the IPCC AR4 models in their paper. Firstly, they note that OHC trends in the models that didn’t use volcanic forcings are consistently higher than the observations. This makes sense of course because each big eruption cools the ocean significantly. For the models that did include volcanic forcings (including the model we used in Hansen et al, 2005, GISS-ER), the match is much better:
This was followed by a figure from Dominguez which showed that the models that included volcanic forcing better matched the revised Ocean Heat Content estimates. In contrast, those based on fictional history with no volcanic eruptions fit less well.
It’s nice to see the models that include volcanic eruptions match observations better than those that ignore the match. After all, hindcasts based on radiative forcings that actually occurred ought to give better results than those that don’t account for things that actually happened.
Let’s ask the obvious question!
Now, if you are intellectually curious, you might ask the obvious question:
What if we filter hindcasts/projections of global mean surface temperatures (GMST) to screen out models that don’t include the volcanic eruptions? Will projection/hindcasts excluding runs based on fictional historical forcings (aka, those without volcanic eruptions) fit observations better? Or…. (play scary music)… will they fit worse?
Why would the intellectually curious ask this? Well, the most prominent projections in the various IPCC reports relate to GMST (global mean surface temperature). GMST is measured, reported, dissected daily on climate blogs and forums all over the world. So, why would anyone who followed the AGW debate not ask this question?
Being intellectually curious, and having downloaded a bunch of data, I decided to do comparison!
As usual, I’ll use the data I downloaded from The Climate Explorer, discussed earlier here.
Compare the fit based on average of 38 runs without filtering inclusion of volcanic eruptions
The figure below compares the monthly GMST as averaged from GISSLand/Ocean, HadCrut and NOAA/NCDC from Jan. 1970-now to the temperature prediction based on the average of 38 GCM runs used by the IPCC in the AR4:
Note the average over the 38 projection/hindcasts runs over-estimates the trend slightly. The projected/hindcast trend over the roughly 38 1/2 years is 1.80 C/century; the observed trend was 1.65 C/century. So, the models overestimate the warming in a period that is mostly ‘hindcast’ by only 10%.
But this included runs with fictional forcing histories. Agung didn’t erupt in the 60s, Fuego didn’t erupt in the 70s, El Chico didn’t erupt in the 80s and Pinatubo didn’t erupt in the 90s. Since Gavin pointed out that using the incorrect forcing should result in incorrect hindcasts (or even projections), we should remove those runs, right?
Let’s look at the volcano and non-volcano runs separately!
Here’s how the hindcast/projections based on volcano-only and no-volcano runs:
The purple represents the average GMST based on the volcano-only runs. Based on the simple average of the 27 runs that actually include the volcanic eruptions that actually occurred, the average trend rises to 1.93 C/century. So, the average run based on models that include volcanic eruptions is 17% too high in the “mostly hindcast”: the agreement is poorer than we get when we include the models with fictional “no-volcano” radiative histories.
In contrast, the trend based on the average over the 11 “non-volcano” runs is 1.48 C/century, which is 10% too low. Since the official IPCC method is based on the average over all models– whether or not they include volcanic forcing– the runs with low trends due to the an-historic forcing that ignores the volcanic eruptions reduces the too high trends for the cases that should be closer too correct.
Variability of 38 year trends
As many readers know, one of the quibbles over testing the IPCC projection of “about 2C/century” for the first few (as in two or 3) decades of this century is related to the effects of volcanic eruptions on the variability of trends. So, are you wondering if the “volcano” runs have larger or smaller variability in trends over the period considered?
Why yes they do! 🙂
The standard deviation of 38 year trends for the cases with volcanic eruptions is ±0.50 C/century. The standard deviation for the runs without volcanic eruptions is ±0.16 C/century– or half that seen for the cases with volcanic eruptions. Of course, this may mean very little as the runs with volcanic eruptions involve different GCM’s than those without volcanic eruptions.
Still, this happens to be consistent with what we find in the observations: When we fit ARMA(1,1) parameter to periods when are not erupting and compare them to those fit to periods when volcanoes are erupting, the variability in trends is smaller when for periods without volcanic eruptions. (This is discussed further here.)
Caveats
Some may note that I did not download every single “hindcast” at the climate explorer. I only downloaded those that continue with A1B projections. Maybe if I downloaded the rest of the runs that include the volcanic eruptions might look better. This would be a bit odd, as it would mean the average run not continued into an A1B projection hindcasts differently than those that continued. But, who knows?
Some will note that the IPCC AR4 method averages over models not runs. In this post, I averaged over runs primarily because it’s easier for “blog work”.
I don’t know what effect this has. However, if we expect the spread in model hindcasts to be the result of “weather noise”, we then weighting by run should give more repeatable results. In contrast, if we expect the spread in hindcasts is largely due to the influence of parameterizations or modeling choices, we averaging over runs makes sense.
(This, btw, the fact that the IPCC AR4 method averages over models is indirect evidence that the authors think the different parameterizations and modeling choices have a larger effect on trend projections than “weather noise”. There are however, other reasons for averaging over models; some may be scientific, others political. )
Some will wonder many things… but at this point I’ll let them ask in comments. 🙂
Wrap up!
But basically the short story is: To remedy the mis-match between Ocean Heat Content (OHC) hindcasts and observations, one should compare models that correctly account for the effect of volcanic eruptions on the radiative forcing. The models with incorrect- anhistorical radiative forcings give poor results for the trend in OHC.
However, when we screen out the model runs with incorrect-anhistorical forcings, the mis-match between hindcasts and observations of GMST from 1970 to the present worsens! Oh well. You win some, you lose some, right?
Fig. 1 has gone AWOL.
Thanks Jorge!
Great blog by the way, I just found it recently.
I did ask some questions about climate forecasting on an AGW blog and just got kind of put down – “see these 2 peer reviewed papers showing how climate forecasting is really good”.
I genuinely can’t understand how a climate forecast can get the attention it does, and this post just confirms it.
What would falsify a climate forecast? You seem to have falsified it by showing that the hindcasting is flawed.
“Doesn’t matter what assumptions we start with, how flawed any data is, if the temperature line matches the historical line up to the present then we can trust the future?”
I had some crazy notion that I could get hold of a climate forecast from last year, and then check the predictions of temperature in a few cities, say August next year, and that would be a good way to check the accuracy of the forecast. But for some reason, I’m told I can’t expect it to be accurate in that way – just the general forecast is accurate.
Can you shed any light on how a truster of climate forecasters thinks?
Won’t the trend be effected not only by how many eruptions, what type (sulfur vs CO2?) , and how strong but where in the timeline they occur? Are there any hindcasts using actuals?
Barry–
The type matters. Fuego, El Chicon, Agung, and Pinatubo were all stratospheric with sulphur. That’s why they are considered when modelers do account for the effect of volcanic eruption. I think Pinatubo was the 2nd largest eruption of the century, and I think El Chicon was the third largest. (I’d have to look up to be sure.) Agung was pretty big– Fuego large enough to “count”.
The hindcasts are based on estimates of either optical depth, dust veil, or whatever specific index that modeling groups used. The intention is to use “actuals”, but of course, there is always some uncertainty in the amount of sulphur injected into the atmosphere, where it was injected, how it fell out, etc.
Steve C–
Thanks! I don’t think you can find papers giving climate forecasts for a few cities. Climate projections are different from weather forecasts.
That said, to be useful, the climate projections should be matching something, some how. GMST for the whole planet over long time periods is the quantity most often projected. That’s why I pay attention to that more than other quantities.
Certainly, if hindcasts of GMST in past periods has an impression of 15%-20%, we can’t expect GCM’s to forecast the temperature in Chicago in 2009. (Modelers don’t claim to be able to project the second.)
Ok, let’s not say cities, let’s pick something else.
My understanding is that climate models are using some kind of finite element analysis, paramaterizing the equations believed to govern all the aspects of climate. And using the correct starting conditions.
So let’s say instead of a city, maybe the model will give a temperature, humidity, cloud cover for several “regions” (boxes in the model). Surely we can look at the output 2 -3 years down the track and compare the model with what actually happened. If it is significantly wrong (enter stats people for definition of “significantly wrong”) then hasn’t that shown the model is flawed?
After all, the best case from then on would be the model using the wrong starting conditions. How can you plug the wrong starting conditions into a finite element analysis model and expect to get the right answers at the end?
So, if that’s right, then ask the climate modeler to hand over the time varying results for each parameter and section of the globe (every 10km^2? 25km^2? 100km^2) and see if they match reality 12 and 24 months from now.
I’m probably missing something. I was involved in finite element analysis a long time ago and we could hind-cast perfectly. Then we ran the next experiment and the model never matched the experiment. So most of us (except the couple of guys employed to do the analysis) thought it was useless. (I should add for completeness that it was semiconductor design not climatology)
Steve
I think it’s more control volume-finite difference and/or spectral of some sort. But yes, they predict the temperature at what one might think of as either small volumes and/or elements.
Climate models don’t necessarily use correct starting conditions. The IPCC runs often “spin up” from more or less incorrect weather way back in 1880 or so. There is some attempt to have approximately correct conditions as one ought to converge faster from almost right weather than wrong weather.
Weather models use correct starting conditions.
Unfortunately, no. This doesn’t show a climate model is flawed!
You really do have to think of “rolling a die” or “flipping coins”. What I try to see is if the models and the real earth seem to have the same statistical properties.
Lucia,
Here’s a comment from 1979,
`”Existing parameterizations of cloud amounts in general circulation models are physically very crude … It must
thus be emphasized that the modeling of clouds is one of the weakest links in the general circulation modeling
efforts”
Charney et al., NRC (1979)
Is it not the case that this situation is roughly the same today almost 30 years later?
How then can GCMs form the basis of any policy decisions about climate change?
Dave Andrews
Have a look at the web-site by Roy Spencer. It will curl your hair. In terms of climate forcing the global climate models dont even get the sign (+ve or neg) correct for some aspects of cloud feedback/forcing.
There is hope that someday soon a renegade young climate modeler may actually incorporate a more realistic feedback/forcing relationship into a GCM just to see what happens.
When this is carried out I think we will see the 100 yr projection for temperature-increase come down substantially, i.e. the projected decadal trend will be lower.
We live in hope.
Rob R
Steve C, what the modellers say of their models Tebaldi, C. and R. Knutti, 2007, The use of the multi-model ensemble in probabilistic climate projections, Philosophical Transactions of the Royal Society A, 365, 2053-2075, doi:10.1098/rsta.2007
is a great paper for understanding the answer to the question you asked. Basically you would first need to wait 30 years for a 30 year model, but that may not be enough time. So for a 100 year projection, you need to wait 100 years. This is the simplistic answer. The paper goes over the general details in a very readable and understandable way. You can find a link at Knutti’s web site.
John, many thanks for your post. I found the paper by Knutti, and will read up.
nice jorge u are so very smart,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, BYE,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,, bye