IPCC Projections Overpredict Recent Warming.
It’s true. Every climate blogger knows it. Global Mean Surface Temperature have gone a bit flat. But is the recent flat trend statistically significant? Well, as my readers know, I took up Roger Jr.’s suggestion and set out to compare IPCC projections to data collected after the projections were made.
Over the weekend, I applied the Cochrane-Orcutt method to monthly GMST data, as reported by four separate groups (GISS, Hadley, UAH and RSS).
This analysis technique permitted me to estimate the best fit line that fits the monthly data and also includes 95% confidence intervals for the slope. I have now compared the trend line that best fits data collected after 2001 to IPCC projections for trends after 2001.
What’s important in the graph?
The short term IPCC projection is show with a red line which I superimposed on the IPCC chart. They predict that during the first 3 decades after 2000, the mean trend will be 2C/century; I indicated that with the straight line. (I extrapolate beyond 2030, so you can see the IPCC expect the trend to increase with thime.)
The shaded area on the IPPC graph shows the IPCC uncertainty intervals around their projected trend.
So, we should expect that if the IPCC trend is correct data trends will fall inside the shaded areas of the graph.
I estimate the empirical trend using data, show with a solid purple line. Not it is distinctly negative with a slope of -1.1 C/century.
But, more importantly, the IPCC projections for the mean trend, as indicated by the red line do not fall inside the 95% confidence intervals for the data. Those confidence intervals are bounded by the two purple dashed lines.
So, both the central tendency of the IPCC projections and the uncertainty intervals the IPCC applied to their projection fall outside the 95% confidence bands for the recent trend based on the data collected after the IPCC projections were made.
So, now, in answer to Roger Jr’s question posed in January:
What behavior of the climate system could hypothetically be observed over the next 1, 5, 10 years that would be inconsistent with the current consensus on climate change?
The current data appears at least somewhat inconsistent with the near term projections by the IPCC. The central tendency of the IPCC projection, m=2.0C/century, falls outside the 95% uncertainty intervals for trends estimed based on data collected since 2001. Moreover, the full uncertainty interval for trends projected by the IPCC fall outside the empirical uncertainty intervals.
Obvious follow on questions
When readers see this graph, I suspect these questions will come to mind:
- What does this mean in terms of IPCC projections: It appears that IPCC projections for the near term trend are high. I don’t know why they are high, but there have been no recent major volcanic eruptions. The climate modelers and NASA say that solar activity can no longer have a significant impact on the trend. “It’s in the pipeline” cannot explain a slowdown in the trend. The effect of thermal mass is to cause the temperature to rise more slowly initially, and then rise more rapidly later as the ocean begins to warm.
- Do I think AGW is ‘over’? Absolutely not.
The confidence interval for this data set are -3.3 C/century < m < + 1.1 C/century. The confidence intervals when I compute the trend with data from 1979 is +1.0 C/century < m < + 2.1 C/century. The newer data is not inconsistent with the longer term trend. (If it were, this would be surprising. I included the newer data when calculating the longer trend!)
Nevertheless, based on data collected after the IPCC projections were first published, the IPCC projections appear to be on the high side. (That said: I would strongly prefer to defer full judgment on the consensus prediction. For many reasons, I prefer to use annual average data rather t han monthly data.).
It has always been a claim of skeptics that long term trends may exist in climate. If they do, any tests based on short series of data are problematic, because it is not possible to detect the true autocorrelation in temperatures from the data set. This applies equally to proving AGW exists and to falsifying it.
- Could different results be obtained with fancier statistical methods? Sure. Possibly someone will perform them.
The main purpose of applying Cochrane-Orcutt or any method to deal with serial autocorrelation, is to get a better estimate of uncertainty intervals. When serial autocorrelation exists, but is not corrected, the serial autocorrelation makes the uncertainty intervals appear to much too small.
It is also worth nothing that even though Cochrane-Orcutt widens the uncertainty intervals significantly, they may still be to small. The reason is that the uncertainty intervals I posted do not account for the uncertainty in the estimate for the serial autocorrelation on the uncertainty in the trend, “m”.
- Did I do lots of thorough fancy checks on this fit? No. I just assumed for the purpose of testing the IPCC model that the data should fit a linear trend and that the scatter around the trend is ‘noise.’ I recognized that the residuals for monthly data are serially autocorrelated and applied Cochrane-Orcutt to the data.
I didn’t check whether the residuals are normally distributed or do any additional checks. The residuals don’t need to be normally distributed to obtain a trendline that minimizes the sum of the square of the residual. However, the distribution of the residuals does matter if we are estimating uncertainty intervals on “m”.
That said, I’m not sure it’s worth a great deal of effort to do a whole lot of checks, at this point. I think it’s better to recognize the uncertainty in the empirical trend is quite large. It’s possible that if the uncertainty in the correlation coefficient were included in the estimate of the confidence intervals, they might widen. If they widened sufficiently, the IPCC projections might end up falling inside the uncertainty band for the data.
(It’s also worth understanding that temperature is not literally expected to vary linearly. However, the near term predictions by the IPCC are nearly linear. So, for the purpose of testing that projection, I assumed the near term trend is linear.)
What next?
Well, now that I think I have a handle on how to do this, I’ll be updating the evaluation rather regularly. We should expect that over the next few years, the confidence intervals on the trend will narrow. I suspect the trend will rise. After all the 30 year trend is positive, and radiative physics do argue strongly for some warming.
Still, as an empiricist, I do like to compare data to projections. So, I plan to do so.
Related posts
For new readers, here are links you might like to read:
- How I applied Cochrane Orcutt:http://rankexploits.com/musings/2008/correcting-for-serial-autocorrelation-cochrane-orcutt/>Correcting for Serial Autocorrelation.
- Why I start comparison in 2001: What Are The IPCC Projections? And How Not to Cherry Pick.
- My initial response to Roger Jr.’s hypothetical question. http://rankexploits.com/musings/2008/what-weather-would-falsify-the-current-consensus-on-climate-change/
References:
Updates:
- March 10: I adding a paragraph to better explain the graph. Later– modified graph for clarity and typos.
- March 11: Link to spreadsheet: GMST data after 2001
- March 11: Images for discussion in comments. (Click to enlarge)
Previous Post:
« Correcting for Serial Autocorrelation: Cochrane Orcutt
Next Post:
The Huge Swing 07-08. Does that affect the results? »
77 Responses to “IPCC Projections Overpredict Recent Warming.”
You can leave a response, or trackback from your own site.
Ian Castles March 10th, 2008 at 2:49 pm
Lucia, Your conclusion that “The current data appears at least somewhat inconsistent with the near term projections by the IPCC” appears to me to be at odds with the recent statement by Australia’s Professor Ross Garnaut (which I’ve quoted on another thread) that “Recent rises in global temperatures [have been] at the upper end of what was predicted [by the IPCC] in 2001.″ Do other readers agree? In particular, do the “real” statisticians agree that you’ve correctly applied the Cochrane-Orcutt technique to ascertain the trend that best fits the monthly data since 2001? (Incidentally, you’ve misspelled ‘Orcutt’ in your chart).
lucia March 10th, 2008 at 2:54 pm
Ian–
The other two people did different analyses, but tell me I did C) correctly. I’ll fix the spelling!
I don’t see how anyone can think rate of change in global temperatures since 2001 is at the uppe end of what was predicted by the IPCC in 2001. If we don’t correct for serial autocorrelation, it’s flat. If we correct, it’s negative. If we look at annual data, it’s flat. How can this be at the upper end?
I guess I’d have to read more of what Prof. Ross said though.
Roger Pielke. Jr. March 10th, 2008 at 3:16 pm
Ian-
This reference you cite comes from Rahmstorf et al. 2007 and is cited on p. 21 of the Garnaut Interim Report. Rahmstorf et al. compare 2001 IPCC predictions with data since 1990. Two things make the analysis here different:
1. The inclusion of more recent data than used by Rahmstorf et al.
2. The 2007 AR4 predictions are a bit higher than those of 2001
Thus the statements here are not at odds with those in the Garnaut Interim Report, and if anything that report and its reliance on Rahmstorf et al. 2007 are now out of date.
Ian Castles March 10th, 2008 at 4:45 pm
Thanks Roger. I think we may be at cross purposes here. The 2001 ‘predictions’ for each of the six illustrative SRES scenarios were shown decade by decade in the tables in Appendix II of the WGI contribution to the TAR. These showed an acceleration of the projected temperature increase in the 2000-2010 decade compared with the (standardised) increase of 0.16 C for the 1990-2000 decade. At the Garnaut Review’s third public forum on 14 November 2007 (”Climate Change: What is the Science Telling Us? Is there a Need to Develop New Emissions Scenarios?”), at which I was a panelist, there was extensive discussion of the most recent report of the Global Carbon Project (Canadell et al in PNAS, 2007) and the claim in that paper that emissions since 2000 have been rising faster than in any of the SRES scenarios. This work was post-Rahmstorf et al. The Garnaut Interim Report modifies the Canadell et al conclusion somewhat (see the last complete para. on p. 15) - I suspect as a result of my comments at the forum and subsequently - but Ross has retained his view that post-TAR developments, including the observed increase in global mean temperatures, have strengthened the case for urgent action.
The statement I quoted on this blog was made by Garnaut to ‘The Age’ before the Interim Report was released (but I agree that he may have been relying on Rahmstorf et al as interpreted by Graeme Pearman, and may not have known of the observed decline in global temperatures in recent months).
I agree with you that if anything the Garnaut Interim Report may be out of date on this point, because of the more recent observational evidence as interpreted in Lucia’s analysis. That is why I am keen to learn of the verdict of expert statisticians on that analysis, and on what it does (and doesn’t) mean. I hope that some experts make submissions to the Review on this important matter.
Ian Castles March 10th, 2008 at 8:02 pm
It’s also worth noting that Lucia’s conclusion has an unambiguous meaning. If I have correctly understood her analysis, she has shown that, using the Cochrane-Orcutt technique to correct for serial autocorrelation, the mean estimate of the rate of change of global temperatures between January 2001 and January 2008 which provides the best fit to the observations during that period, and using the average of the four series as published by the respective sources, is minus 1.1 C/century. One can debate the reasons for this result, whether or not the number of observations is sufficient, et cetera, but at least the conclusion has a clearly defined meaning.
By way of contrast, the statement in the WGI contribution to AR4 that “Six additional years of observations since the TAR (Chapter 3) show that temperatures are continuing to warm near the surface of the planet” (Chapter 9, p. 683) is ambiguous. The ordinary meaning of this sentence, as constructed, is that the observational record shows that temperatures CONTINUED to rise DURING the six additional years following the TAR. But it appears from the full context that the meaning that the lead authors had in mind was that during these six years the world was warmer on average than in various earlier periods such as the first 50 years of the instrumental record or the first decade of the twentieth century or some other period. This says nothing about the trend DURING the six-year period concerned. It’s surprising that the lead authors of a chapter in a scientific report, and the various government and expert reviewers, were satisfied with such a fuzzy statement.
John V March 10th, 2008 at 8:53 pm
Hmmm, my numbers don’t quite match yours. I hope you don’t mind a question or two to track down the discrepancies:
Using monthly data, I get the following global trends from Jan 2001 to Feb 2008:
GISS: +0.83 C/century
HadC: -0.55 C/century
RSS: +0.41 C/century
UAH: -0.07 C/century
AVERAGE: +0.16 C/century
However, when I compute the trends from Jan 2002 to Feb 2008 I get:
GISS: -0.29 C/century
HadC: -1.67 C/century
RSS: -0.91 C/century
UAH: -1.71 C/century
AVERAGE: -1.14 C/century
From your article I’m not sure if you intended to start in Jan 2001 or Jan 2002. I believe your previous articles stated that Jan 2001 would be the start date. But it looks like you may have started in Jan 2002. Is that correct?
It’s amazing the difference a single year can make.
John V March 10th, 2008 at 8:59 pm
…or maybe I’m way off base and Cochrane-Orcutt can actually change the slope by that much.
lucia March 11th, 2008 at 4:20 am
John,
I’ll post figures of the least squares and Cochrane-Orcutt today. I have been known to make mistakes in my life, but looking at my spread sheet, my calculations do start in 2001. Feb 2008 data aren’t fully in yet, so I don’t have Feb 2008 in the calculation.
lucia March 11th, 2008 at 5:52 am
JohnV,
Yes. I do think it’s important to start in 2001. Picking and choosing years gives the analyst free reign to pick whatever the heck they want and screws up the basis for stating confidence intervals. (When there isn’t much data, it also screws up the trend– but generally shifting them within the confidence intervals.
Here are thumbnails of the charts. You can click to see larger images.
As you can see, we do get different trends for different data.
I added the spreadsheet above so you can check my numbers. (I downloaded the data from Watt’s site– as noted in my earlier post. Obviously, if something got corrupted in between, that would cause problems. But, I do start from Jan 2001.)
On your observation of the choice of year: YES. Right now, switching by a year makes a difference in conclusions. So, when Feb and March data come in, it may turn out that, while this “falsifies” in terms of hypothesis test, the later data show it just happened to be the 2σ event. (These happen– 5% of the time!)
Nevertheless, as Roger pointed out, for something to be called science, it must be at least hypothetically possible to falsify. Otherwise, it’s pseudo-sciece. Clearly, the IPCC does make projections that are at least hypothetically falsifiable, and I think it’s important to show how one would falsify and demonstrate what happens.
John V March 11th, 2008 at 6:28 am
lucia, thanks for the extra details.
I was exhausted when writing last night and hope it didn’t come across as adversarial. The negative trend caught my attention because I’ve never seen that before.
lucia March 11th, 2008 at 6:37 am
JohnV,
You didn’t come off as adversarial. The negative trend stunned me when I saw it too!
Anyway, you and I both discussed checking the correlation in the residuals in the past. So, I happen to know you are very interested in this, as am I.
In all honestly, I’d love you to check my numbers. I think the issue of “how much warming” is important. You should also note that this result doesn’t say AGW is falsified; the upper uncertainty intervals clearly do include 0C/century.
The result says 2 C/century lies outside the bounds. So, if it’s correct, it gives us information to bound estimates and also to gauge how well the IPCC project and/or draws estimates their own uncertainty interval.
FWIW….. I’m waiting for February to come in. I have GISS, RSS and MSU data; I’m waiting for Hadley. I notice the numbers announced by the various agencies are susceptible to change the first few weeks after they are announced.
Still, I’ll be posting as soon as Hadley is in. I do know, based on “fiddling” that the “uptick” on the normal Temperature vs. Time plot looks small. It’s bitter on the Cochrane-Orcutt chart, because that takes into account the issue of correlation. Still, the new number is not going to reverse the negative value. That requires a change in the weather.
I’m also trying to learn some other methods– ARMA etc.
John V March 11th, 2008 at 7:25 am
I’ve been able to confirm that we were working on the same numbers. (A small step, but definitely important). I will try to find time to understand C-O so I can check the rest. Today does not look good though…
BTW, now that we’ve cleared up the coincidence of OLS trends from Jan2002 matching C-O trends from Jan2001 I’m pretty confident that your calculations are correct. The spreadsheet *looks* right.
lucia March 11th, 2008 at 7:40 am
JohnV,
I found NOAA numbers, and I’m adding that to the average.
Also, I need to learn the other techniques. I helpful anonumous tutor says I should look at various other methods too.
BarryW March 11th, 2008 at 8:57 am
Lucia,
Could you post your graphics in a larger size? Some of us don’t have the resolving power to see the details.
lucia March 11th, 2008 at 9:38 am
BarryW–
I make them smaller to save bandwidth. (I guess I’m used to blogging about knitting where the majority of my visitors always had the slowest connections.)
The full size graphics are also in the spreadsheet. So, if you download that, you can open in Excel and see the graphic. The link is in the “update” section.
lucia March 11th, 2008 at 9:44 am
Oh— Also, the graphic in THIS post started out in the IPCC document. It’s already blown up compared to their document. I added lines so we could have some hope of reading it.
Unfortunately, the IPCC isn’t to good about providing the sorts of numbers that make it easy to falsify of validate easily. To do a full t-test, I would need to actually read the values at the top and bottom of their uncertainty intervals to estimate what they are saying the standard error in their prediction is.
lucia March 11th, 2008 at 9:53 am
oh… heh. I just realized, you may mean the thumbnails in the update. Click on those, they’ll open to new windows with larger images.