IPCC Projections Overpredict Recent Warming.
It’s true. Every climate blogger knows it. Global Mean Surface Temperature have gone a bit flat. But is the recent flat trend statistically significant? Well, as my readers know, I took up Roger Jr.’s suggestion and set out to compare IPCC projections to data collected after the projections were made.
Over the weekend, I applied the Cochrane-Orcutt method to monthly GMST data, as reported by four separate groups (GISS, Hadley, UAH and RSS).
This analysis technique permitted me to estimate the best fit line that fits the monthly data and also includes 95% confidence intervals for the slope. I have now compared the trend line that best fits data collected after 2001 to IPCC projections for trends after 2001.
What’s important in the graph?
The short term IPCC projection is show with a red line which I superimposed on the IPCC chart. They predict that during the first 3 decades after 2000, the mean trend will be 2C/century; I indicated that with the straight line. (I extrapolate beyond 2030, so you can see the IPCC expect the trend to increase with thime.)
The shaded area on the IPPC graph shows the IPCC uncertainty intervals around their projected trend.
So, we should expect that if the IPCC trend is correct data trends will fall inside the shaded areas of the graph.
I estimate the empirical trend using data, show with a solid purple line. Not it is distinctly negative with a slope of -1.1 C/century.
But, more importantly, the IPCC projections for the mean trend, as indicated by the red line do not fall inside the 95% confidence intervals for the data. Those confidence intervals are bounded by the two purple dashed lines.
So, both the central tendency of the IPCC projections and the uncertainty intervals the IPCC applied to their projection fall outside the 95% confidence bands for the recent trend based on the data collected after the IPCC projections were made.
So, now, in answer to Roger Jr’s question posed in January:
What behavior of the climate system could hypothetically be observed over the next 1, 5, 10 years that would be inconsistent with the current consensus on climate change?
The current data appears at least somewhat inconsistent with the near term projections by the IPCC. The central tendency of the IPCC projection, m=2.0C/century, falls outside the 95% uncertainty intervals for trends estimed based on data collected since 2001. Moreover, the full uncertainty interval for trends projected by the IPCC fall outside the empirical uncertainty intervals.
Obvious follow on questions
When readers see this graph, I suspect these questions will come to mind:
- What does this mean in terms of IPCC projections: It appears that IPCC projections for the near term trend are high. I don’t know why they are high, but there have been no recent major volcanic eruptions. The climate modelers and NASA say that solar activity can no longer have a significant impact on the trend. “It’s in the pipeline” cannot explain a slowdown in the trend. The effect of thermal mass is to cause the temperature to rise more slowly initially, and then rise more rapidly later as the ocean begins to warm.
- Do I think AGW is ‘over’? Absolutely not.
The confidence interval for this data set are -3.3 C/century < m < + 1.1 C/century. The confidence intervals when I compute the trend with data from 1979 is +1.0 C/century < m < + 2.1 C/century. The newer data is not inconsistent with the longer term trend. (If it were, this would be surprising. I included the newer data when calculating the longer trend!)
Nevertheless, based on data collected after the IPCC projections were first published, the IPCC projections appear to be on the high side. (That said: I would strongly prefer to defer full judgment on the consensus prediction. For many reasons, I prefer to use annual average data rather t han monthly data.).
It has always been a claim of skeptics that long term trends may exist in climate. If they do, any tests based on short series of data are problematic, because it is not possible to detect the true autocorrelation in temperatures from the data set. This applies equally to proving AGW exists and to falsifying it.
- Could different results be obtained with fancier statistical methods? Sure. Possibly someone will perform them.
The main purpose of applying Cochrane-Orcutt or any method to deal with serial autocorrelation, is to get a better estimate of uncertainty intervals. When serial autocorrelation exists, but is not corrected, the serial autocorrelation makes the uncertainty intervals appear to much too small.
It is also worth nothing that even though Cochrane-Orcutt widens the uncertainty intervals significantly, they may still be to small. The reason is that the uncertainty intervals I posted do not account for the uncertainty in the estimate for the serial autocorrelation on the uncertainty in the trend, “m”.
- Did I do lots of thorough fancy checks on this fit? No. I just assumed for the purpose of testing the IPCC model that the data should fit a linear trend and that the scatter around the trend is ‘noise.’ I recognized that the residuals for monthly data are serially autocorrelated and applied Cochrane-Orcutt to the data.
I didn’t check whether the residuals are normally distributed or do any additional checks. The residuals don’t need to be normally distributed to obtain a trendline that minimizes the sum of the square of the residual. However, the distribution of the residuals does matter if we are estimating uncertainty intervals on “m”.
That said, I’m not sure it’s worth a great deal of effort to do a whole lot of checks, at this point. I think it’s better to recognize the uncertainty in the empirical trend is quite large. It’s possible that if the uncertainty in the correlation coefficient were included in the estimate of the confidence intervals, they might widen. If they widened sufficiently, the IPCC projections might end up falling inside the uncertainty band for the data.
(It’s also worth understanding that temperature is not literally expected to vary linearly. However, the near term predictions by the IPCC are nearly linear. So, for the purpose of testing that projection, I assumed the near term trend is linear.)
What next?
Well, now that I think I have a handle on how to do this, I’ll be updating the evaluation rather regularly. We should expect that over the next few years, the confidence intervals on the trend will narrow. I suspect the trend will rise. After all the 30 year trend is positive, and radiative physics do argue strongly for some warming.
Still, as an empiricist, I do like to compare data to projections. So, I plan to do so.
Related posts
For new readers, here are links you might like to read:
- How I applied Cochrane Orcutt:http://rankexploits.com/musings/2008/correcting-for-serial-autocorrelation-cochrane-orcutt/>Correcting for Serial Autocorrelation.
- Why I start comparison in 2001: What Are The IPCC Projections? And How Not to Cherry Pick.
- My initial response to Roger Jr.’s hypothetical question. http://rankexploits.com/musings/2008/what-weather-would-falsify-the-current-consensus-on-climate-change/
References:
Updates:
- March 10: I adding a paragraph to better explain the graph. Later– modified graph for clarity and typos.
- March 11: Link to spreadsheet: GMST data after 2001
- March 11: Images for discussion in comments. (Click to enlarge)
Previous Post:
« Correcting for Serial Autocorrelation: Cochrane Orcutt
Next Post:
The Huge Swing 07-08. Does that affect the results? »
77 Responses to “IPCC Projections Overpredict Recent Warming.”
You can leave a response, or trackback from your own site.
steven mosher March 11th, 2008 at 11:21 am
Hey JohnV,
Do you think we can agree on a common format for these monthly anomalies. Some of the GISS stuff I have is
in a table ( year by month). The RSS and UAH stuff is in a table of year/month vectors. I prefer the hadcru approach,
a vector… ALSO, I just found out that Openoffice supports R extensions in its spreadsheet. FWIW
What do you think? Your Opentemp would be a good place to host it, being Open and all.
Something Like
Year Month UAH RSS GISS NOAA HADCRU
1850 1 xx xx xx xx xx
….
2008 2 xx
Not sure how you want to code missing data for those records that only go back to 1979 or 1880.
Anyway it would give folks who want to load it into a spread sheet and play around a good resource.
Or Lucia could do it.
Also Lucia, I’m unclear why you mean by Averaging these difference temperature indices, since they measure different
things. Not saying its wrong, but just asking for clarity
lucia March 11th, 2008 at 11:38 am
stevemoscher–
It would be nice if these were all in similar formats. But, they aren’t. Still, I find as long as all I’m looking for is monthly anomaly data, I find it and getting it into the spreadsheet isn’t anywhere the time sink that learning new ways to process data is. (Of course, getting the real temperatures is not so easy. OTOH, it’s really easy to find the little on line lectures telling us we shouldn’t want it. And that if we think we want it, we should realize we shouldn’t want it. No…. it’s not what we should want.
)
As for archiving: I’m not an appropriate person to be an archivist. It doesn’t suit my personality. . . Take my word for this! I’d be wretched.
The fact is, these data also sometimes sift. UAH found an error earlier this year, SteveMcYntire found errors in the NASA processing which caused them to fix an error. Who ever the archivist is, they will need to keep track of this stuff, and communicate it. I’m just not the person to do it.
What I’m doing currently is getting the data from an archive, slapping it into my spreadsheet, and after doing the analysis, uploading the current version of the spreadsheet to my wordpress logs. That way, if I screw up, at least precisely what I used is traceable. (I wasn’t even in this habit back in January. . .) I’m trying to remember to state where I got the data in the blog post when I report the data. But really, that’s about all.
lucia March 11th, 2008 at 1:09 pm
For those wondering, I hunted down the NOAA monthly values and added them to my analysis. The 95% confidence bands for the trend, with NOAA added to GISS (Land/Ocean), Hadcrut, UAH and RSS are:
-3.0 C/century < m < 1.1 C/century.
The best estimate based on this data is -0.9C/ century. So, NOAA to the average, the IPCC projections are inconsistent with this data.
steven mosher March 11th, 2008 at 4:24 pm
Lucia,
Dont sell yourself short. you are the pefect person to do this work for the rest of us. and we would
all be ever so grateful…
Aint working huh? rats
steven mosher March 11th, 2008 at 4:28 pm
How are you averaging? NOAA, GISS and HADcru are not independent measures, and UAH RSS measure something else..
Chillguy33 (1 comments.) March 11th, 2008 at 9:25 pm
Is there any connection between IPCC and Oil For Food?
The efforts appear to operate with remarkable similarity. Is it the same folks?
TIA
PaulM March 12th, 2008 at 5:59 am
steven mosher,
Anthony Watts has done that at least back to 1979, see his very useful file 4metrics_temp_anomalies.txt which is in almost exactly the format you suggest. Let’s hope he or somebody else keeps it up to date as the new data comes in. Of course you are right that just averaging GISS HADCRU UAH RSS is a bit crude, but I think that’s what Lucia is doing and it seems as good as anything else.
Lucia,
I’m afraid I have doubts that these numbers are significant. If we just look at the simple least-squares trend I get -0.11 C / century from Jan 2001 - Jan 2008 (which I think agrees with the b in your spreadsheet). But if I start in Feb 2001, I get -0.28, and starting in March 2001 gives -.40. So the trends are all over the place, if you change your start by just one data point. But they are all less than +2 C/century, so maybe your conclusion still holds. Expert statistician needed - have you asked Briggs?
lucia March 12th, 2008 at 6:45 am
PaulM-
The error bands stated in the blog post are:
-3.3 C/century < m < + 1.1 C/century
The values you give (-1.1 C/century, -0.28 C/century and -0.4 C/century) all fall in the stated uncertainty intervals which were calculated using this method.
I think what you are seeing is precisely what those calculated uncertainty bands mean: We expect the value of “m” to vary “all over the place”– but within [-3.3 C/century, 1.1 C/century] based on this specific data.
So, yes, if what I’m claiming is true, I’d expect the various different choices to result in precisely the sort of scatter you are finding.
We could ask Briggs. He knows more than I do about this stuff. But, no, I haven’t asked him.
The Huge Swing 07-08. Does that affect the results? | The Blackboard March 12th, 2008 at 8:05 am
[...] Comments: IPCC Projections … [...]
aaron (4 comments.) March 12th, 2008 at 10:12 am
Chillguy33, yeah. They are both run by the UN.
Roger (1 comments.) March 12th, 2008 at 7:25 pm
Not surprised the Garnaut report is out of date already. It was never going to contadict from Stern.
If Garnaut was relying on an interpretaion by Graeme Pearman it would certainly be one of “dangerous warming”, of which Graeme has been an advocate for many years.
Ah well, it’s only taxpayer’s money.
At least the new PM has said it’s one of many inputs to climate policy. Hopefully the issues raised here will also also be considered before $zillions are committed to little or no effect.
Some of you may be interested in gustofhotair.blogspot.com which does statistical stuff with local data
Ian Castles March 12th, 2008 at 8:26 pm
Thanks Roger. I need to make clear that my comment about Garnaut relying on an interpretation of Pearman was based on the following statement by the Garnaut Review itself:
‘This section draws largely on the research of the Intergovernmental Panel on Climate Change (IPCC) as interpreted and presented by Dr. Graeme Pearman’ ( Issues Paper 3, p. 2. The section is headed “The Global Impacrs of Climate Change”).
lucia March 13th, 2008 at 6:13 am
Ian and Roger–
I plan to check that particular issue when I have time. It seems to me that there is lots of rhetoric on both sides, often not supported by any number of analysis. (Certainly, politicians rarely give citations when interviewed. I don’t actually expect them to do so.)
One of the difficulties is tracing down precisely what was ‘predicted’ in some of these documents. I’ve quickly examined the TAR, and didn’t find projections stated quite as clearly as in AR4. (I may find them yet, but so far… just not quite a clear.)
Some of the earlier documents are “unfalsifiable” in the scientific sense because they only predict things one could never even hope compare against empirical data.
For example, there is no really good empirical data to compare predictions of steady state climate sensitivity as a function of doubling of CO2. Schwartz (2007) gave it a good shot looking at post 1880. Hansen gave it a shot looking at glacial vs. interglacial. But to get really good data, we need to double the C02 and let the planet sit there for a while. Obviously, this experiment can’t be done! All we have for that is models.
Ian Castles March 13th, 2008 at 7:45 am
Lucia, I think that you’ll find the TAR ‘predictions’ you’re looking for, decade by decade, in Appendix II (”SRES Tables”) of the contribution of Working Group I (”Climate Change 2001: The Scientific Basis”). These Tables show that, since 2000, emissions have increased more rapidly than in most of the SRES scenarios. This leads in turn to a faster-than-projected rise in atmospheric concentrations, in forcings and in the model average surface temperature (see Table II.4).
One way of looking at this is to imagine that, when representatives of the world’s governments met in Shanghai to approve the WGI TAR in January 2001, they were so seized with the urgency of the climate change emergency that they agreed to implement drastic immediate reductions in emissions of greenhouse gases. What would the consequence have been?
As the SPM of the WGI contribution to AR$ explains, “Even if the concentrations of all greenhouse gases and aerosols had been kept constant at year 2000 levels, a further warming of about 0.1 C per decade would be expected’ (p. 18, second sentence of emphasised paragraph).
If this had happened, the slashing of emissions would by now be being hailed as a great success. According to your analysis, the UPPER LIMIT of the increase in temperature since January 2001 has been at the rate of about 1.0 C/century - i.e. 0.1 C per decade. But this has been achieved in the face of an ACCELERATION of the growth of emissions, not a drastic reduction!
Raniers or Maraschino? Accusations of Cherry Picking and Climate Change. | The Blackboard March 14th, 2008 at 10:06 am
[...] SteveUK– I can’t fix your… IPCC Projections … [...]
William (2 comments.) March 15th, 2008 at 11:38 am
I think you’ve misunderstood the IPCCs pic. It doesn’t include natural variability uncertainty: http://scienceblogs.com/stoat/....._front.php
lucia March 15th, 2008 at 11:54 am
William,
My analysis assumes their error bars don’t include natural variability.
steven mosher March 15th, 2008 at 12:12 pm
Lucia,
What has to happen in 2008 for the IPCC to get back on track?
what has to happen in 2009? in 2010?. there is clearly an envelope of futures
that puts the IPPC back in line with their projections. How probable is that?
I think I have a way of looking at this…empirically. lazy saturday
lucia March 15th, 2008 at 1:07 pm
SteveM,
Interesting questions. It would take some fiddling to do active speculating! Maybe monday. But I may address the issue both William and Boris brought up: Weather noise, and how that relates to this. I think that may be more important than trying to figure out what hypothetical weather needs to happen to get IPCC back on track in by the end of the decade.
steven mosher March 16th, 2008 at 8:45 am
Lucia, I looked at the weather noise in a rather odd stupid way. I made every possible 30 year time series
from GISS 1880-2007 (t1-t30,t2-t31, etc etc ) I figure every 30 year sequence has got some climate trend
and some weather noise. Then I made a big spaggetti graph, grafting every 30 year sequence onto 2001.
A huge spread in data. ennormous variability over a 30 year span ( using an observationalist perspective)
Take away. The IPCC projections are based on GCM. GCM underpredict the variability of weather on short time
scales, and have not been tested on longer time scales.
Raven March 16th, 2008 at 12:15 pm
Lucia, Steven,
Here is a model output that specifically includes ‘weather noise’:
http://en.wikipedia.org/wiki/I.....bution.png
From the link:
This is from a Meehl et al., 2004 paper which is referenced numerous times in AR4. This should allow you quantify the weather noise built into the IPCC projections.
lucia March 16th, 2008 at 12:28 pm
Raven,
If the bands are interpreted to include weather, then the IPCC did an even worse job at predicting the central tendency.
I will be discussing the weather vs. climate issue either on Monday or Tuesday. My hubby is mostly monopolozing the computer doing our taxes and our inlaws taxes today. And yesterday we were having a big get-together.
The weather/climate issue William discussed is a non-issues as the standard statistical techniques distinguish between year to year variability and trends. That’s actually the whole point of using this technique.
Boris March 16th, 2008 at 12:39 pm
I have a few more questions:
1. If using the TAR estimates, why did you decide to use a graph from AR4? It’s not a big deal, but I am curious.
2. Where do you get the 2C century number for IPCC TAR’s projection? Does this number have error bars or a shaded area like the graph shown? If so, is it based on the ensemble mean? (The shaded area in the graph shown is based on the ensemble mean.)
3. Does the IPCC discuss “weather noise” in relation to their projection?
4. Does “during the first 3 decades after 2000″ actually start in 2001? I assume so, but there’s a bit of ambiguity there.
Frank O'Dwyer (19 comments.) March 16th, 2008 at 1:17 pm
Lucia,
“The weather/climate issue William discussed is a non-issues”
His point re your (mis)interpretation of the shaded area is an issue. The shaded area in the graph represents uncertainty in the climate sensitivity of the models. You are interpreting them as a confidence interval around the projected trend itself, but they aren’t and they don’t purport to be that. If I have understood correctly, they represent confidence about the model prediction, not in the model prediction. If I’m right then you would need to include this further uncertainty in order to compare like to like. And if you did, it seems plausible that the top of your error range would overlap the bottom of the IPCC range.
“IPCC projections
overpredictoverlap recent warming” is not such a snappy headline but you could still in principle falsify the projections with more data, and it would still be an interesting exercise. However I don’t know how you would go about quantifying the additional uncertainty in the prediction.lucia March 16th, 2008 at 2:09 pm
Boris,
I use the AR4 because that’s the most recent one. Also, the AR4 itself, though published in 2007, calls what they are doing projecting the climate after 2000. (As you can see, the graphic includes data through 2000.)
The reason for the lag is the projections were not done fresh during 2007 and were published ain a previous document. The AR4 itself includes a graph with the 2C/century number and reiterates it in text several places.
As far as I am aware, the IPCC does not specifically discuss weather noise, but if you know differently, let me know. My understanding of their document is they discuss climate, as that is the issue of interest. In a previous post, I discussed what weather, measured using annual average data is consistent with the climate trends they project. In this post, I discuss the climate trends that are consistent with the weather measured at monthly intervals that we actually experienced.
Yes, the 3 decades are the first three of this century, and as you can see on the graph, that is the trend for this century, not the last decade of 1900. The whole thing is encapsulated in the graphic. As you can see, the graphic includes data up to and including 2000; the projections are for climate that occurs after.
For what it’s worth, I picked the start date for analysis before I obtained the monthly data and before I learned how to do Cochrane-Orcut. And obviously, if the temperature trends turn up, the falsification won’t “stick”. But right now, that’s what the projected climate is insistent with the recent string of weather we actually experienced.
lucia March 16th, 2008 at 2:26 pm
Frank:
William says this:
I agree with William natural variability (weather noise) is not included in the IPCC projection. I agree with him the error bars are not supposed to constraint the year-to-year variation.
I agree with William that if the uncertainty intervals included natural variability, the uncertainty intervals would be larger.
So, William is confirming what I believed when I did my analysis: The projections do not include natural variablity. My analysis assumed this.
My analysis is specifically designed to detect which underlying trends (climate) are consistent with the weather we actually experienced. This technique is used in calibration, quality control and all sorts of other areas. It’s taught to undergraduates in engineering and science curricula, though the most painful lessons are generally driven home when doing laboratory experiments.
It is not clear to me why William thinks this makes my analysis incorrect. However, I’ll elaborate more, since this seems to be an area some have difficulty with.
As for your comment, could you explain whatever you think is the difference in ” confidence about the model prediction” and “confidence in the model prediction”?
I can’t image what the difference might mean, quantitatively.
I’m also a bit bemused that William says “Its not absolutely clear what the error bars are to me,”.
If a climate scientist at RealClimate doesn’t know what those uncertainty intervals mean, what was the public supposed to take away from that graphic? (That said: The text of AR4 says what they mean. I know because I read it. They apply to uncertainties in predicting climate.
)
Frank O'Dwyer (19 comments.) March 16th, 2008 at 3:00 pm
Lucia,
“As for your comment, could you explain whatever you think is the difference in ” confidence about the model prediction” and “confidence in the model prediction”?”
William put it like this:
We have a family of curves going into the future, of the nature of dT=CO2*f*t, for various values of f, where t is time. IPCC is graphing that. Missing from that is the weather noise, which corresponds to the throw of the dice for each year
In other words, there is some uncertainty about what the models actually predict in the first place - or maybe this is better stated there is a range of curves that could be used for prediction, and it is uncertain as to which one is the best one to pick if you want to make a prediction. It is this range of different model climate sensitivities that the shaded area represents. The mean is chosen as being the prediction likely to be closest to obs, but (since the climate sensitivities aren’t certain) it could be that a more accurate prediction lies in the shaded area. If that’s correct then the 95% confidence interval (shaded area) you’re using is just about the climate sensitivies.
The problem is that nothing about the process described above says what the error in the prediction (the difference between prediction and obs) is likely to be. It is just the process for making the prediction in the first place. (My understanding of it, anyway. I could be wrong.)
In any case, if I am right, then your use of the central trend in the IPCC graph should still be OK but you’re using the wrong error bars around their predicted trend, because the error bars you need aren’t stated at all in that graph.
lucia March 16th, 2008 at 3:39 pm
Frank-
My purple lines are the 95% confidence intervals for climate trends that would be compatible with the weather we have had.
The IPCC shaded areas are their range of projections for the possible climate trends compatible with the different climate sensitivity. Yes. That’s how I interpret them.
Their range of projections for climate trends are not compatible with the weather we have had.
The IPPC trends are not weather predictions– they are climate predictions. So, of course they don’t tells us how well they predict weather. They tell us what the underlying trend– as expected given a range of climate sensitivies– is supposed to be.
So, in other words I have interpreted them exactly as you say I should.
Boris March 16th, 2008 at 5:00 pm
Lucia, the text from the figure you’ve used says:
So the IPCC is clear on what the shaded area means, but this graphic is not the most suitable for your comparison.
Also, since it is a mean, the internal variability should be, to a large extent, averaged out.
Mark Bahner (3 comments.) March 16th, 2008 at 6:42 pm
I don’t agree that the IPCC made any predictions in 2001. They had a bunch of scenarios, and the temperature rises associated with those scenarios. But none of the scenarios matches the trends in the various climate forcing agents. For example, all the scenarios have higher atmospheric methane concentrations than have been observed, and most or all had higher black carbon emissions than have (likely) occurred.
However, on the cooling side, the most scenarios overestimate the SO2 emissions that have (likely) occurred.
If I say that there’s going to be a thunderstorm if the high temperature tomorrow hits 90 deg F, and a foot of snow if the high temperature doesn’t get above 30 deg F, it’s not possible to say how accurate those “projections” were, if the high temperature is 70 deg F tomorrow.