Recently, David Stockwell has been discussing Rahmstorf et al 2007, focusing on how accounting for uncertainty in the data would affect interpretation of the IPCC TAR predictions relative to data. David has been focusing on issues related to the end points; today I want to discuss something much simpler. How does the uncertainty in determining the “true” temperature for 1990 affect the conclusions in Rahmstorf.
The reason this “true” temperature is important is that, IPCC projections are not given based on a predifined anomaly, but rather to some “true” value of temperature from 1990. The method of determining that value is not discussed in the TAR, nor is that temperature provided. (Though, one might think the annual global means surface temperature for 1990 might have been known by 2001, which the TAR was published.)
So, it turns out that analyst must shift for themselves and figure out the “true” temperature in 1990, and then figure out if the IPCC projections were high or low.
I think I will show qualitatively, how various methods of determining the “true” temperature for 1990 give different results. The results differ sufficiently to affect the conclusions in Rahmstorf. One could conclude the TAR predictions were high, low or in the vicinity of correct.
In fact, since there is no standard method for picking the temperature for 1990, it’s a cherry picker’s dream.
The organization of this post will be as follows:
- Show the TAR projections are relative to some annual average global mean surface temperature for 1990.
- Show that Ramhstorf did “slide” to some true value.
- Discuss different different methods of estimating the “True” value for the annual average global mean temperature result in different conclusions.
- Discuss uncertainty about the estimate of the true value.
The TAR Projections: Relative to 1990
The TAR projections are provided in several places. In my opinion, the clearest figure is 22, from the technical summary. It is clearly evident that the Hadley and GISS temperatures measurements are normalized to make T=0 in 1990, see below.
Of course, it is rather well known that the temperature anomalies are not zero in 1990. This is of little consequence, as one can simply adjust the predictions to place the T=0 anomalie in 1990. (This can either be one by shifting the experimental data down, or shifting the projection up.)
However one thing is clear: To do the shifting, we must know the temperature in 1990. Otherwise subtraction becomes difficult.
Rahmstorf “pins” Measurements to some temperature for 1990
It turns out that Rahmstorf7, do shift the data.
This is the Figure from Rahmstorf et al 2007. Temperatures are annual averaged global mean temperatures from HadCrut and GISS. Measured temperatures are rebaselined to match the “true” temperature in 1990. That’s why the TAR predictions and the “true” temperature for 1990 line up exactly in 1990.
Rahmstorf et al. describes his method of shifting, but does not state the numerical value or discuss any uncertainty in the shift. So, let us assume, after doing some statistical computations based on the noisy data, they concluded the temperature anomaly in 1990 was 0.2C and shifted that much.
That’s actually fine as far as it goes. One must “slide” the data to place them on the same baseline as the projections.
However, there is a difficulty. It must be realized that calculated value for the “True” temperature in 1990 is based on noisy weather data. So, there is uncertainty associated with that value. Whatever value one calculates, there should be some ±0.xC associated with that value. If “real true’ temperature is 0.25C, then the entire IPCC curve wold be shifted relative to the data in the figure above.
To many words? Well, lets look at pictures.
They then discuss the significance of the results, ignoring the possibility that there might be some
Results with different “true” values
The simplest possible value one might select for the “true” GMST in 1990 might be the actual measured temperature in 1990. Rather than make my own graphs, I’m going to super-impose TAR trends, and uncertainty triangles on charts created by David Stockwell
Here’s an example of what one might conclude if they happened to like GISS data and decided the “true” temperature for 1990 was the actual measured temperature. That the true value might be the actual value is not inconceivable to some.
So, taking that to be the “true” value, I “slid” the IPCC projections to the 1990 value:
Ref:Dave Stockwell
If the figure above, the dark purple line is a central tendency of 1.5C/century. The fuzzy lines are 2.0 C/century and 1C/century. (It is possible to show these are roughly the uncertainty bands, though the true ones curve.)
Comparing the data to the curves, I think many people would conclude the TAR projections look high.
So, this is one way to do the “slide and eyeball” method. The reason these results differ from Rahmstorf is I “slid” to the true temperature for 1990.
But many would argue that’s a poor choice because weather are noise. So, in that case, one might want to apply some sort of trend analysis. We could fit a straight line (over which years?). Or do some sort of fancy fit, or what not. (Rahmstorf selected a particular version of “fancy fit”.)
Here’s what we might see if we picked a “CaterpillarSSA” with 11 year embedding:
Alternate figure, starting from 2000.
Examining this, we see the temperatures near the end of the series are outside and above the values projected for climate trends. But then, values earlier are outside and below. This is the opposite of what we concluded above.
Of course, we must also bear in mind we are eyeballing a graph, and comparing the noisy weather data to predictions for smooth underlying trends. So, it’s an apples to oranges comparison. But never mind. Let’s just forget about the fact that we are comparing weather to trends and move on.
Let’s answer the question a bunch of reader who likely climate-blog-war posts are asking: What the heck is a “caterpillar SSA” fit? Why not just use a fit we see all the time? Why not use get the “real” 1990 tempeature using the method GISS and Hadley use at their sites?
Ok. Here we go: Below I’ve shifted the TAR projections onto the Hadley GISS splines applied to monthly data.
In this graph, the red wiggly lines are GISS and Hadley splines applied to monthly data. The purple is are the 1.5C/century projections surrounded by 2C/century and 1C/century bands.
Now, the IPCC TAR projections, since 1990 don’t look all that bad, as determined using the “calibrated eyeball method”.
Sure, the weather data pop outside the uncertainty intervals. That’s expected because the IPCC projections are for climate trends, not weather. (And this is why David and I estimate climate trends that could consistent with weather data to compare to IPCC predictions of climate variables.
Uncertainty intervals.
Some will have noticed that I showed three methods of calculating the “slide”. Picking the value for 1990 is “high”, using the fancy fit is “low” using the Hadley GISS method is “medium”. Each gives different conclusions about the match between IPCC projections and data. This happens because there is a great deal of uncertainty associated with this method of testing the skill of IPCC projections.
Also, I showed annual only data, and monthly data. The monthly data gives a sense of the noise in the data; the annual data does not. So, using annual data makes the slide and eyeball method worse because the naive user can’t immediately see the noise, and account for its important in a tacit way.
So, if we are to use this eyeball method, it might be best to communicate the impact of the choice of the method of estimating the true temperature for 1990– which dictates the amount of the slide, and also communicating the uncertainty associated with this. It’s also best to avoid obscuring the noise in the data; plotting annual average data tends to obscure that.
Had the uncertainty in “the slide” been discussed in Rahmstorf, most would likely decide that we can’t tell if the IPCC TAR projections were high low or whatever. Had monthly data been shown, readers might have developed some sense of the uncertainty in verifying TAR projections using this method (which is a poor choice). Using the slide and eyeball method we need more than 17 years to figure out anything.
However, the fact that we need so much data for “slide and eyeball”, doesn’t mean we need that much data for other methods. “Slide and eyeball” is crude. That’s why it’s not used in formal hypothesis testing. Other methods, like t-test, taught in undergraduate curricula in many fields, give more reliable results when comparing noisy data to predicted climate trends.
GISS data estimation of global temps reliable?
http://data.giss.nasa.gov/cgi-bin/gistemp/do_nmap.py?year_last=2008&month_last=3&sat=4&sst=1&type=anoms&mean_gen=03&year1=2008&year2=2008&base1=1951&base2=1980&radius=250&pol=reg
http://climate.uah.edu/
sorry hard to believe
In Rahmstorf et al (2007) it is stated that ‘The global mean surface temperature increase (land and ocean combined) in both the NASA GISS data set and the Hadley Centre/Climatic Research Unit dataset is 0.33 C for the sixteen years since 1990 …’
In the light of (a) the above analysis, (b) previous related posts on this blog, (c) David Stockwell’s posts at Niche Modeling and (d) comments by others at both blogs, including Stefan Rahmstorf’s replies to David, does any reader care to defend this unqualified statement by Rahmstorf and his co-authors?
I am surprised tamino or anyone else hasn’t rushed in to defend Rahmstorf7.
Ian, isn’t the issue here that the increase of 0.33 degrees in 16 years is not unexpected in a stochastic trending up at 0.2C/decade? Rahmstorf7 asserts that 0.33C in 16 years means something significant: that the climate system is responding more quickly or something. Statistics suggest that such increases might be expected occasionally. That is, the esteemed Science reviewers of IPCC chapter authors are indulging in cherry picking and feel free to disregard basic stats tests.
David, I think it’s a bit more complicated. Consider first the following paragraph in the SPM of AR4:
‘Since IPCC’s first report in 1990 assessed projections have suggested global average temperature increases between about 0.15 C and 0.3 C per decade for 1990 to 2005. This can now be compared with observed values of about 0.2 C per decade, STRENGTHENING CONFIDENCE IN NEAR-TERM PROJECTIONS’ (p. 12, EMPHASIS added).
So far as the 1990-2000 period is concerned, the IPCC ‘projection’ of the increase in temperature should NOT be compared with observed values because the latter reflect ACTUAL forcings, not projected levels. In particular, the TAR’s projections of emissions, hence concentrations, hence forcings, did not allow for the steep fall in emissions (= concentrations) of sulphate aerosols that is estimated to have occurred in the 1990s. The most comprehensive estimates are those by David Stern in ‘Chemosphere’, which the critics of ‘The Great Global Warming Swindle’ DVD were quick to cite for the 1950-75 period in explanation of the slight cooling that occurred during this period (see Steve McIntyre’s post ‘Risk Management Solutions Ltd. and the 37 Professors’ at Climate Audit, 3 May 2007).
But if the rise in sulphur emissions in the third quarter of the 20th century serves as an explanation of the cooling, it must be presumed that the (unanticipated) fall in these emissions in the 1990s would have contributed to the spurt in warming at the end of the 20th century. For further detail, see my Comment 1250 on the ‘Rahmstorf et al 2007: Where does their figure come from?’ thread on this blog, and the immediately succeeding Comments.
Moreover, the IPCC ‘projections’ for the 1990-2000 decade were standardised averages of a subset of the projections: see, for example, my post to the ‘You can’t make this stuff up’ thread at ‘Prometheus’, 20 March 2008 at 10.50 pm. Rahmstorf et al (2007) appear not to have understood the basis upon which the ‘projections’ published in the TAR were produced.
So for the first 10 years of Rahmstorf’s 16-year period, there’s no reason why the relatively rapid warming should have strengthened confidence in the IPCC projections for the early decades of the 21st century. The proper comparison with ‘recent climate observations’ is with what the IPCC projections would have shown if the decline in sulphur emissions had been anticipated, which they weren’t. It’s surprising that 200 governments overlooked this elementary point when they approved the text of the AR4 SPM.
There’s no reason why the post-2000 observations of GMST increase should strengthen confidence in near-term projections either, for reasons that emerge clearly from Lucia’s analysis. Your observation about the Rahmstorf7 assertion that the climate system is responding more quickly (than projected) is right as far as it goes, but it can be taken further: Lucia’s analysis indicates that ‘IPCC projections overpredict recent warming’ (title of the March 10 post). This means, I think, that in this respect the climate system is responding LESS quickly than projected (subject to the limitations of the analysis that Lucia has spelled out in her posts).
Have you seen the article ‘Sceptics will have their day’ by Mark Lawson of the ‘Australian Financial Review’, which was published by the Australian e-journal ‘On Line Opinion’ last Thursday? It appears that Lawson has also been corresponding with Stefan Rahmstorf about the article that appeared in Science last May. It’s to his (Rahmstorf’s) credit that he has responded to both of you, but I suspect that he’s trying to defend the indefensible.
This is a postscript to my previous comment in response to David Stockwell’s question about Rahmstorf7 and the reason why IPCC authors feel free to disregard basic statistical tests.
In an invited contribution (‘ A Critique of Wood on Global Warming’, 26 July 2006) to the blog of Australian economist John Quiggin, Dr. Roger Jones of CSIRO, an IPCC Coordinating Lead Author, criticised the Wegman Committee for recommending that ‘evaluation by statisticians should be standard practice’ in climate change research, including in the ‘application-for-approval process’ (Report, p. 6):
‘That they want to involve statisticians in ongoing work is interesting. What level of education in statistics does one need to have? Skill in statistics does not mean a better understanding of science or even uncertainty… Ian Castles … attacked the IPCC SRES scenarios on statistical grounds, without showing that the underlying assumptions relating population and energy use were in fact incorrect. The idea of using statisticians without training and a publication record in the relevant science, or as an integral part of a larger team should not be given air.’
I’d be off-topic if I were to pursue Dr. Jones’s claims about my critique of the IPCC scenarios, but it IS relevant to the present discussion that Jones was one of several CSIRO IPCC Lead Authors who responded to the IPCC’s recent invitation to comment on the Panel’s future – and that, In that context, he offered the following criticism of AR4:
‘The difference between projections made in “model worlds†and how the real world was tracking were (sic) overlooked [by the IPCC], allowing misleading projections to be made for the early 21st century especially (e.g., 0.2 C per decade for the next few decades – WE ARE ALREADY ABOVE THIS RATE AND ACCELERATING)’ (IPCC-XXVIII/INF ‘Future of the IPCC – Compiled comments from Governments, Authors, Organizations and Bureau members’, p. 93, EMPHASIS added).
I don’t know of any hard evidence that supports the latter contention: I suspect that, like the Garnaut Climate Change Review, Jones has relied on, and perhaps misinterpreted, the findings of Rahmstorf et al (2007).
Ian, If I could clarify a bit, having not made the IPCC a subject of deep study. First there is the scenario with sulphate negative forcing predicting medium rates of warming say. Then, the sulphate levels don’t materialize. But the rates of warming match the predictions, and so the IPCC claims victory. And nobody cares. Right?
It seems your focus is on the frailty of claims on the ‘acceleration’ of warming. I hadn’t previously been aware of this particular extreme position.
I hadn’t read the article, but I did and he seems to have the same view of Rahmstorf7 – sloppy and probably wrong. As to my statements about Rahmstorf7, they are distinct from what I think about the climate system. That is, when I say the most reasonable interpretation is fluctuating about a long term trend, that doesn’t mean that I think the system is fluctuating about a long term trend, and will continue to do so. I am trying to be precise about a particular analysis and the logic or illogic of the conclusions drawn from them, in order to come to some sort of resolution.
Part of my interest in the uncertainty of end points of the series too, is that it is a general question that might interest say a financial analyst smoothing time series, because my blog has a more general audience. Lucia’s attach is more specific to the paper. You can understand not anticipating a lot uncertainty at the end of trends, but the arbitrary nature of the problems Lucia points out should have been obvious.
It strikes me that these people are so convinced they are right and so there is no need for stats. I personally think they are ‘completely wrong’ about GHGs but have been keeping quiet about it until I had proof, at least to the point where I would not be shifted into the climate crank box. Skeptics shall have their day, indeed. I’ll be working on this next.
Anyway, more on this later. I have to finish a paper on optimal shapes of earthen roadside drainage structures right now.
David, On your first para: Yes, that’s right. The slight cooling in the 3rd quarter of the 20th century, notwithstanding increasing forcing from GHGs, has to be explained. So rising emissions of sulphate aerosols were offered as the explanation. When comprehensive (but not necessarily accurate) annual estimates of these emissions become available, the IPCC milieu latched on to the 1950-75 estimates to support the validity of forcings in models, but ignored the 1990-2000 estimates from the same source because the sharp downturn in emissions in those years doesn’t help to explain anything. This seems to me to be a clear case of confirmation bias.
On ‘acceleration’ of warming, the use of that word may be an extreme position, but the claim that ‘It’s worse than we thought’ seems to be a pretty constant refrain. In recent days, Lord Stern has been reported as saying:
“Emissions are growing much faster than we’d thought, the absorptive capacity of the planet is less than we’d thought, the risks of greenhouse gases are potentially bigger than more cautious estimates, and the speed of climate change seems to be faster,†[Stern] told Reuters at a conference in London … Stern said on Wednesday that increasing evidence of the threat from climate change had vindicated his report, published in October 2006. “People who said I was scaremongering were profoundly wrong …â€
Well, global emissions aren’t growing MUCH faster than ‘we’ thought, if by ‘we’ he means the modellers who produced the emissions scenarios used in the TAR and again in AR4. At most, these emissions are at the upper end of the SRES range. And atmospheric concentrations of GHG aren’t increasing more rapidly than ‘we’ thought either. And, putting those two things together, it’s not apparent that ‘the absorptive capacity of the planet is less than we’d thought’ either. More generally, I’m not at all sure that the new evidence that’s become available since the Stern Report DOES vindicate the positions taken in that Report, and in any case I wish he’d be more specific about what new evidence he has in mind.
The Garnaut Interim Report is however explicit in citing Rahmstorf7 as the source of their concern that ‘it’s more urgent than we thought.’ That’s why I’d like to see some authoritative support for your and Lucia’s criticisms of the paper. Thanks Lucia for providing me with the opportunity of giving an extended explanation of my position.
Ian–
You are entirely welcome!
I think the Rahmstorf paper proves that the IPCC needs someone who knows some statistics on board.
As to this:
All I can say is the fact that Rahmstorf has 7 authors, got published in Science, and that Stefan Rahmstorf criticizes Pielke for not considering the uncertainty in the data, while holding up Rahmstorf is proof positive these climatologists need some statisticians on board. (Or at least someone who is willing to open a book, discover that some techniques exist and apply them.)
At a minimum, if statistics isn’t your field, and you need to do determinations against data, you should:
* Use standard published methods many people know. (Unless you are doing something really novel, methods that appear in textbooks are preferred over methods documented in a single obscure papers.)
* Learn how to determine the uncertainty according to that method.
* Learn to check whether the assumptions for that method are violated.
* Learn the difference between β and α error.
* Learn how to spot obvious flim-flam when it’s going on. (That is, strawmen, cherry picking, idiosyncratic choices of analysis techniques fished out of lightly cited articles.)
I think the climate scientists could do this if they wished. But it doesn’t occur to them. They think “slide, eyeball, muse” is fine. It’s not a good way to figure out if the IPCC projections/predictions were ‘good’. (It’s even worse if we start discussing about the distinction between projections and predictions.)
I would too. The time for comments is closed at Science. Garnaut didn’t put my submission up. Perhaps a good thing, but I might ask them why this week. Maybe E&E? Boy, those Garnaut submissions are a mixed bag! http://www.garnautreview.org.au/CA25734E0016A131/pages/submissions-general-submissions
I predict that this statement will appear on RC, tamino, Rabett, and a number of other places to prove that David is not a Certified Climatologists and is thus not qualified to be investigating Climate Change issues. 🙂
Hmm… I had to figure out what was wrong with my plugin. The links to David’s comments were ‘ rel=”nofollow” ‘ and he’d commented more than 3 times!!!! ( I forgot that I need to adjust the limit on number of characters. . .
Dan, There does seem to be a strong tendency to argument by authority. I liked the recent twist on argument by accusations of mis-spelling. I have to say, that on the one hand when I know little, I do tend to acknowledge that authorities likely know more about their subject than others. But on the other hand the moment someone (say B) says ” ‘A’ is right because ‘A’ is an expert, I suspect that person ‘B’ does not know enough to actually distinguish between a correct and incorrect statement or claim. Otherwise, if person ‘B’ knew anything at all, they’d advance an argument based on substance.
The recent “argument by complaining about spelling”…. well, let me tell you, that sort of things always makes me suspect the one arguing by spelling really can’t identify a good counter argument!
Rahmstorf replies:
Sweet. More later.
@1996
Interesting that the Garnaut Review has not published David Stockwell’s submission. Mine (very brief) neither. I hope that this is a temporary omission. It would be unprecedented, I think, to omit submissions from the public record.
David–
Stefan’s response is truly lame! It’s as bad as the previous defense by accusing critics of spelling his name badly.
He’s got to know that private individuals aren’t going to spend the thousands on peer reviewed articles, and we can certainly comment. Also, that Rahmstorf paper shouldn’t have been published in the first place. Why should people waste their money publishing one that decrees “We compared the data to the IPCC projections using a method similar to Rahmstorf, but accounting for uncertainty, and no sane person would draw any conclusions!”
Stefan is just going to have to get used to the fact that people will read his paper, comment and point out it’s dreadful. If he’s going to blog, he ought to learn to stop defending by telling people to spend thousands of dollars out of pocket to gain the right to criticise.
That paper is dreadful, and it shows nothing. The fact that it was published reflects backly on Science and the seven authors who can’t tell how dreadful their own paper is.
Hi Peter,
I talked to a rep. from the Garnaut Committee and he said they just hadn’t got around to it. He said they had over 1000 submissions to go through. If ever you need proof this is a hot button issue that is it.
Anyway, I revised my submission in light of interaction with Rahmstorf and I resubmitted. I put in links to lucia’s blog, mine and realclimate for supporting documentation.
Apparently, none: wasn’t it Mann who admitted “I am not a statistician”?
Does this mean that lack of skill in statistics makes one a BETTER climate scientist?
Several things here: 1) Very few “peer-reviewed” climate-based journals will accept papers written by statisticians, 2) Climate science is based in multiple fields and statistics applies to all of them, and 3) If their “larger team” HAD a statistician, we wouldn’t be having this argument (or have the papers we do).
lucia,
I think the appropriate response from Stefan would be to redo his analysis correctly. This would be an honorable thing to do.