Raniers or Maraschino? Accusations of Cherry Picking and Climate Change.
As many climate warming junkies are aware, a number of people began to notice a sort of “flatness” in the Global Mean Surface Temperature in the past decade. Way back in December, in attempt to prove the recent trend in temperature has no statistically significant meaning, the Blogger formerly known as Tamino did the following:
- Generated a series consisting of 100 years of data with a known trend of 0.18 C/century and ± 0.1 C. (This is approximately equal to the trend over the past two decades).
- Manually picked out a specific data point that happened to be “the high” relative to the trend.
- Showed that if you intentionally pick an “high” outlier out of a series of containing 100 points, you there will be a negative trend afterwards and
- Concluded that the recent downtrend is somehow statistically normal.
Of course, it wasn’t too difficult for the Blogger formerly known as Tamino to do the analysis he wished to do. The difficulty is that his little “proof” hardly supports his conclusion that recent trends in GMST fall inside the bounds of what might be expected based on any statistical measures.
Yet there are those who, in comments at blogs, are suggesting Tamino’s so called proof demonstrates that looking at recent trends to assess the rate of climate change is cherry picking.
The reality is: If the IPCC is correct about its projections, the recent trends are highly unusual. There is sufficient recent data to support this statement using standard statistical techniques. These are the exact same techniques used to support the contention that the non-zero warming trend predicted by AGW is supported by empirical data.
What do statistics tell us about Tamino’s little test?
Based on analysis of 30 year trends, statistics tell us that the standard error in the 10 year trends is approximately 1.1 C/century. If the trends are normally distributed, we expect that 5% percent of all possible trends will have slopes less than zero. Clearly, if temperature trends stay rock steady at 1.8 C/century, if you sift through 100 years looking for one with a negative trend, you can find one. 2
The ease with which one can pick out a negative 10 year trend might seem to be Tamino’s point: If the real trend is 1.8C/century with a variability around the trend of 0.1C, if you sift through 100 years of data applying absolutely no rule to limit your choice you can fish out a 10 year series with temperature trends that are less than zero.
But now one must ask this: Has anyone other than Tamino used Tamino’s technique to pick cherries? Has anyone used that technique to support the contention that the recent trend in temperatures is not exactly racing skyward?
No. Or to put it more emphatically: Absolutely not.
Constraints
In the first place, those who have examine recent trends are always compelled to select the most recent year as their endpoint: these include David Stockwell, Basil, and me.
Not one of us selected any old, entirely arbitrary, 10 year string out of 100 years to provie our point. Why would we? What could we possibly hope to demonstrate by showing there was or was not a flat point in the 10 year temperature trend trend back in 1940? Quite likely, no one is stupid enough to attempt that rather interesting method of cherry picking. Though, should someone do so, the proof that such a choice is silly now exists online.
In reality, what those who examine recent trends have done is to constrain our choice of strings by some rule.
David Stockwell, uses the most recent 10 years. David correctly points out that the only free variable is the length of the data set used for analysis. So, to examine shortish trends he selects 10 years. Ten years may be somewhat arbitrary, but it has the advantage of being a round number. If one asked, “Why not use 11 years? Why not 9?” the answer is: “David limits himself to round numbers. ”
I prefer to eliminate even this latitude. When testing the fidelity of IPCC predictions, I limit myself to data collected after they make their projections. This means that, due to statistical uncertainty, I assume the IPCC predictions are true until data sufficient to prove the IPCC projections false trickles in. (I actually anticipated the data would support their projections. But data are data, so what is one to do?)
Basil uses a standard statistical technique to select his start year.
Let us see what each of us are finding:
- David Stockwell examines the recent year to year drop. Using the IPCC definitions of “likely”, and “very likely”, he determines that the recent 10 year trend is inconsistent with the IPCC predictions of 2C/century. Depending on the data set used, the inconsistency ranges form “medium likelihood” to “very likely”.
- Limiting myself to data arriving after the IPCC made their projections — and there by limiting myself in a way that entirely prevents cherry picking– I find that, using an average of four measurement data sets, I find the recent trend is inconsistent with IPCC projections of 2C/century. Using the IPCC defined terminology for level of confidence, the central tendency of 2.0 C/century is “virtually certain” to fall outside the bounds of the data. The lower bound of their uncertainty intervals is “extremely likely” to fall outside the bounds of the data.
But notice that whether we use David’s method or mine, we each get similar, though not identical, results. The only difference in our conclusions is the confidence with which we falsify the most recent IPCC projections.
- Basil, guest posting at WattsUpWithThat?, is applying the Chow Test to see if there appear to be ‘breaks’ in the GMST temperature series. He identifies a structural break at 2001:2002. The Chow Test is a standard test, and Basil reports the statistical significance of the break. Using IPCC terminology, according to the test, he find it is at least “very likely” that a break in the temperature trend occurred at 2001:2002.
Using his own, more cautious terminology, Basil thinks the likelihood the statistical technique found a real hinge point falls between “probably” and “very likely”. He also notes that application of the techniques doesn’t explain the cause for the break.
We do not yet know what Basil will conclude about the predictive ability of the IPCC or statistical significance of any temperature trend. But I think it is safe to say that he will explain the basis for his choice of “start date”.
What do we find? And conclude?
No matter which major temperature measuring group we examine, or which reasonable criteria for limiting our choices we select, it appears that possible that something not anticipated by the IPCC WG1 happened soon after they published their predictions for this century. That something may be the shift in the Pacific Decadal Oscillation; it may be something else. Statistics cannot tell us.
It may turn out that this something is a relatively infrequent but climatologically important, feature that results in unusually cold weather . Events that happen at a rate of 1% do happen– at a rate of 1%. So, if recent flat trend is the 1% event, then 30 year trend in temperatures will resume.
For what it’s worth: I believe AGW is real, based on physical arguments and longer term trends, I suspect we will discover that GCM’s are currently unable to predict shifts in the PDO. The result is the uncertainty intervals on IPCC projections for the short term trend were much too small.
Of course, the reason for the poor short term predictions may turn out to be something else entirely. It remains to those who make these predictions to try to identify what, if anything, resulted in this mismatch between projections and short term data. Or to stand steadfast and wait for La Nina to break and the weather to begin to warm.
But back to the accusations of cherry picking?
Did Basil, David or I use the Tamino Technique for cherry picking?
Some may not like the results we are posting. But, clearly, we did not cherry pick to obtain them. We each constrained ourselves to starting from the most recent year and working back and each of us limits ourselves further by an additional constraint.
Some may dispute our methods for selecting our start dates. But everyone needs to select a start date and end date for statistical tests somehow. Some may dispute the notion of comparing IPPC predictions to short-series of data. But the uncertainty intervals of hypothesis tests are expressly designed to account for the larger uncertainty in small data sets.
To those who wish to decree the time periods are cherry picked based on results they dislike, I would ask: How would you select your start dates to test the accuracy of IPCC predictions? If their choice to support the possibly high IPCC near term projection of warming at a rate 2C/century is not based on the invention of the thermometer but rather appears to be selected based on a point near a relative minimum in the temperature series, I would say:
“Your highly processed Maraschinos look delicious. May I taste one?”
Footnote
1. Of course, it may be difficult to find a negative trend, announce it, and then see it immediately by precipitous drop. The recent, rather dramatic drop in temperature was noted many, including, instapundit, Andrew Bolt, Tim Blair and Michael Ascher. Of course that drop occurred after Tamino’s post showing the flatness was not unusual.
Previous Post:
« So, if tubs DON’T over flow, we can ignore AGW . . . right?
Next Post:
Inadequate Reasons to For Suggesting the Falsification of IPCC Projections Doesn’t Apply. »
40 Responses to “Raniers or Maraschino? Accusations of Cherry Picking and Climate Change.”
You can leave a response, or trackback from your own site.
Boris March 14th, 2008 at 2:08 pm
First off, I’m not accusing you or anyone else of cherry picking.
But consider this. We know that 1998 was a year when internal variability, specifically a strong el nino, was adding heat to the atmosphere. Further, we know that currently internal variability, specifically a la nina event, is taking heat from the atmosphere. Given that knowledge, we must also admit that if we were going to see one of those unusual trends–the 1 in 20 or 1 in 50–then the conditions are currently optimal.
So, treating this as a pure numbers problem can mislead us into premature rejections of the IPCC projections. We can also look at the physical process of internal variability to see if there is a weather explanation rather than a climate explanation.
Raven March 14th, 2008 at 6:17 pm
Boris says:
The fact that this period coincides with the drop from a solar max to a solar min could also indicate that the effect of the sun on climate has been underestimated. IOW - the trend may tell us that the IPCC projections are actually wrong because they overestimated the influence of CO2.
jorge a. March 14th, 2008 at 7:46 pm
dear lucía:
before “The ease with which one can pick out a negative 10 year trend…” there is a number “2″ but only one footnote… a mistake??
off topic but, why do you say: “Blogger formerly known as Tamino”???
thank you!
great blog
lucia March 14th, 2008 at 8:53 pm
Boris–
I know you aren’t accusing me of cherrypicking. This accusation is being bandied in comments at other blogs– and not specifically at me, but at those who are looking at recent data at all.
I’ll get you the specific %s tomorrow. But bear in mind: my time period does not include the 1998 El Nino. That happened before the IPCC made it’s predictions. I’ve chosen to simply examine their projections.
And yes, there is the possibility this is the 1 in 100 event. That said: Typically, the 1 in 100 cool events come after major volcanic eruptions which are known to veil the sun. So, I’m not entirely sure that La Nina is explanatory. Nevertheless: Yes things that happen 1 in 100 times, do happen 1 in 100 times.
lucia March 14th, 2008 at 9:03 pm
First: my point in this article is that in picking our time periods, our start end years are constrained. No one is simply hunting for any randome set of years that might contradict some particular trend.
Also, who says 1975-1985 falsifies the known trend? And which know trend? Do you mean the real 1975-1985?
lucia March 14th, 2008 at 9:06 pm
Jorge:
Oppps. I edited out the second footnote! Tamino renamed himself “Hansen’s Bulldog”. He blogged about his decision here
Ian Castles March 14th, 2008 at 10:22 pm
Lucia and Boris,
To this non-scientist, it is far from clear what the IPCC is attempting to ‘predict’. On the one hand, the authors of Chapter 10 of AR4 (’Global Climate Projections’) say that ‘Uncertainty in prediction of ANTHROPOGENIC climate change arises at all stages of the modelling process’ (s. 10.5.1, p. 797, EMPHASIS added). On the other hand, they say in the same paragraph that ‘[S]ome sources of future radiative forcing are YET TO BE accounted for in the ensemble projections, including those from … variations in solar and volcanic activity’ (ibid., EMPHASIS added).
If the purpose is to predict anthropogenic climate change, why try to incorporate the proximate effects of variations in solar and volcanic activity? And if these effects are considered to be beyond prediction, which would be understandable, why say that they are ‘yet to be’ accounted for?
Frank O'Dwyer (19 comments.) March 15th, 2008 at 4:08 am
Lucia,
“Showed that if you intentionally pick an “high” outlier out of a series of containing 100 points, you there will be a negative trend afterwards and”
Actually he also showed that if you do it unintentionally it will be the case.
“The difficulty is that his little “proof” hardly supports his conclusion that recent trends in GMST fall inside the bounds of what might be expected based on any statistical measures.”
Strawman. That conclusion appears nowhere in the article you link.
You have apparently entirely missed the point of the article which is not only that some denialists really have cherry-picked 1998 as a high point (a real-world example is shown in Tamino’s article), but that they will use ANY short-term noise which appears contrary to AGW. If there is not a 10-year trend they will use a 7-year trend and they will if necessary use a 1-month trend. If there isn’t a 1-month low they’ll use a single cold day in Chicago or a picture of some snow. If you find yourself having to defend against accusations of cherry-picking (and I’m not accusing you of such), it’s those guys you need to thank.
Your examination of the recent trend is interesting, but you have done yourself no favors here by misinterpreting Tamino’s illustration as a ‘proof’, entirely missing the point of it, pretending that it argues for conclusions that it does not, and going on to both misrepresent and minimise his efforts (’little test’, ‘little proof’).
You’ve apparently also missed the point of this paragraph:
The right approach is to look for the trends, not the wiggles, and to apply statistical significance testing to determine whether they’re real changes in the system or just accidental fluctuations. There’s always noise mixed in with the signal, and disentangling the two can be very tricky. But it can be done.
According to you, this is what you’re attempting. Why then do you respond to Tamino’s illustration as though it were an attack on you?
For a tamino post that is a bit more relevant to your discussion, see this.
lucia March 15th, 2008 at 4:10 am
Boris–
I checked the specific number to answer how of this event would be expected to occur if the IPCC 2.0 C/century were correct, and we had the weather variability that actually occurs. It’s less than 0.006 (0.6%) So, it’s the 6 out of a thousand event according to this particular statistical model, with it’s associated assumptions. (Normally distributed slopes and what-not.) This uses a 2-tailed distribution. Depending on the wording of the hypothesis you had in mind, a 1 tailed would be more appropriate, in which case, the probability would be even lower.
I usually prefer to just pick a confidence interval and give a conclusion rather than going through and trying to find the ‘break even’ confidence interval for a particular conclusion.
Ian–
It’s true they didn’t entire account for volcanic eruptions. But, volcanos didn’t erupt during this period. The sun didn’t do anything truly odd during the recent 7 years.
Whatever caveat may be included, I think it’s still worth seeing how the projections compare to data. And, for all the caveats included, what the specific short term projections are is easier to find in the AR4 than the TAR! (I’m sure they are in there, but I like being able to find numbers in the Technical Summary, and the guide for policy makers. I figure numbers that are crystalized down in those document are the actual projections as interpreted by the public an policy makers. Detail in appendices is great, supports and helps interpret what the TS and guide for policy makers mean, but the front matter is there for a reason!)
lucia March 15th, 2008 at 4:18 am
Frank Dwyer.
No. Tamino showed that if you run a random series with 100 points and a trend, a “high” point will exist. (This is well known.) Then if he hunts down that “high” point and intentionally makes it his high point, there will be a negative trend after that.
His decision to hunt down the “high” point was entirely intentional. When he did that, he applied zero restrictions to his choice. He could have selected any point he wished, he intentionally applied the high one, and did his analysis.
Getting it “accidently” would involve:
* creating a criteria for picking 1 specific 10 year trend (for example, the last one in the series),
* running a one and only one series of 100
* Then opening his eyes and saying: Whoa! Look! The data just, somehow, accidentally, worked out so that there is a down tick in the final 10 years!
The probability of the downtick occuring in one specific series he predefined is much lower than the probability he create a series, hunt through the data and find one.
He cherry picked, but others were not doing what he claimed.