What weather would falsify the current consensus on climate change?
In January, Roger Pielke Jr. asked
What behavior of the climate system could hypothetically be observed over the next 1, 5, 10 years that would be inconsistent with the current consensus on climate change?
Meanwhile, Gavin Schmidt explained that we really can’t look at short-term weather to assess models. While Gavin’s answer is perfectly correct , it falls short of answering Roger Jr.’s specific question: Roger would like to know if we can state, in advance, any sort of weather that is inconsistent with the ‘current consensus’ on climate change.
In fact, no matter how variable weather, it is possible to answer Roger’s question– provided it is made a bit more precise. In my opinion, the plot below answers the question:
“What trend in GISS Land/Ocean temperatures over the next 5, 8 or 10 years, would be inconsistent with the most recent IPCC projections of climate change?”
This question can be answered, because it nails down a metric– GISS Land/Ocean, and specifies the “current consensus” with projections that are published, and, so, knowable. The answer is a bit complicated, since the IPCC provided a range of projections, described the probabilities in somewhat vague terms and has different projections for both short and long terms trends.
In my opinion, the short answer to the question is: If, the weather is such that an ordinary least squares fit to GISS Land/Ocean data for the next decade shows any negative trend, this would be inconsistent with the IPCC’s short term projection for temperature which appears to be 2.0°C per century.
A trend of 2.0°C per century, evidently, represents the mid-point of model projections based on SRES scenarios. So, in some sense, falsifying a 2.0°C per century predicted trend would amount to falsifying the current GCM’s prediction for the central tendency, while accounting for the range of uncertainty introduced by weather.
Details about IPCC spread.
In reality, the IPCC provides a range of predictions/projections for warming trends, the magnitude of the projected temperature trend varies both according to time frame and emissions scenario,. The most commonly cite values are increases expected to occur during the 21st century: The full range, including the most extreme scenarios is 1.1°C/century to 6.4°C/century. The most probable range is said to be 1.8°C/century to 4.0°C/century, with, evidently 2.0C/century representing the midpoint of SRES model projections.
After performing a statistical analysis, I plotted the trend in GISS Land/Ocean temperatures trends calculated using ordinary least square (OLS) using data collected over the next 5,8 and 10 years that I think would be inconsistent with the IPCC range of projections in climate change. That plot is shown below.
In today’s blog post, I’ll explain how to read the graph. Some time soon, I’ll explain how I concocted the graph. I may also post a few other graphs and explanations of statistical tests next week.
How to read the graph.
First, assume we believe “consensus” position is that global mean surface temperatures (GMST) will rise at a rate of 2.6°C per century — the midpoint of the “most probable range” suggested by the IPCC. Further, let’s assume we believe temperatures are already rising at 2.6°C per century. Then, of course, one would expect that the probable rate of rise for GMST over the next decade is about 2.6°C a century, right?
But, as Gavin correctly pointed out, due to weather noise (and volcano eruptions and other unpredictable phenomena), even if the underlying trend due to AGW is, 2.6°C/century the trend in temperature averaged over decade might be higher or lower than 2.6°C/century. But, how much higher or lower?
This question can be answered using some statistical reasoning- and I’ll explain that later. For now, let’s assume my method is sound and read the answer of my graph.
Draw a vertical line from “2.6°C/century” — the supposed “consensus” value, stop at the yellow line corresponding to a 10 year trend, and then read “0.6°C” to find the lowest possible trend over 10 years that is ‘not consistent with’ 2.6°C/century being true.
So, according to my graph, a mean temperature rise of 0.6°C obtained by Ordinary Least Squares (OLS) using data collected during the upcoming 10 years is “not inconsistent” with a prediction of 2.6°C/century.
In contrast, if the measured trend over the upcoming 10 years, is less than 0.06°C over 10 years, then this will be inconsistent with an assumed trend of 2.6°C/century .
(For those wondering, a measured trend of 0.6°C a century measured over 10 years also ‘not inconsistent with’ the real trend being 0°C/century. Weather being what it is, all sorts of trends are ‘not inconsistent with’ a variety of hypotheses. )
Other values read from the chart are also provided below.
| Rise per Century | Minimum rise per century over period. | ||
| 5 year | 8 year | 10 year | |
| 1.1°C | -6.3°C | -1.9°C | -0.9°C |
| 1.8°C | -5.6°C | -1.2°C | -0.2°C |
| 2.0°C | -5.4°C | -1.0°C | 0.0°C |
| 2.6°C | -4.8°C | -0.4°C | 0.6°C |
| 4.0°C | -3.4°C | 1.0°C | 2.0°C |
| 6.4°C | -1.0°C | 3.4°C | 4.4°C |
For any given predicted or projected rate of temperature increase show in the left hand column, you can read the minimum rate of temperature increase, calculated using ordinary least squares, measured over 5, 8 or 10 years periods is ‘not inconsistent’. Measured rates below the appropriate value to the right are inconsistent with the projected value.
Oh, by the way. My analysis includes variability due to volcanic eruptions. So, and I’ve made choices that make it difficult to falsify. I’ll explain how I did this later. Few people want to read math unless they are interested in understanding a result, but I strongly suspect others will want to understand what I did so they can figure out if there are any analysis choices they would dispute.
So, is there any weather that could falsify IPCC projections?
Yep, in principle, some sorts of weather could falsify IPCC projections.
Unfortunately, it’s not very easy. Weather, even when averaged over 10 years is so highly variable, that the range of weather that is ‘not inconsistent’ with even 10 years worth of projections is quite large.
In particular, if the trend over the next 10 years falls below “0°C/century” of warming, this will be inconsistent with 2°C/century of warming, which appears to be the midpoint of model predictions for the next two decades. (See Wikipedia; look for discussion of “SRES” projections.)
It is a bit ironic however, that the break even point for ‘falsifying’ the IPCC claim happens to correspond precisely to no-warming. This was, as it happens, a complete coincidence, but there you go!
Previous Post:
« Spatial Variations in GMST Really DO Matter when Estimating Climate Sensitivity.
Next Post:
Can IPCC projections be falsified? Sample calculation »
44 Responses to “What weather would falsify the current consensus on climate change?”
You can leave a response, or trackback from your own site.
Roger Pielke. Jr. February 25th, 2008 at 11:22 am
Interesting analysis.
So to be clear — if there is no warming 2001-2007, then if this continues through 2010 then model predictions of IPCC AR4 (2.0 degrees/century) can be said to be falsified?
lucia February 25th, 2008 at 12:33 pm
Wrong number of years, but otherwise yes. If there ordinary least squares for 2008-2017 has a slope of ‘0 C/century’, that contradicts the IPCC AR4 2C/century central tendency. I look at future values only, so I start in 2008.
If this happens, those who really don’t want to give up a theory could say IPCC wasn’t wrong their documents’ permits 1.1C/ century, but the higher values are out.
If we try the OLS sooner- running 2008-’15, then the trend needs to be -1C/ century.
I’ll be posting how I calculated this when I can. Likely tomorrow or Wednesday.
The explanation of how it’s calculated will clarify, and if anyone reads my blog, discussion might help people understand where there are some assumptions. But the method I’m using is the one I’d use when planning an experiment. There is always a little wiggle room after the experiment because numbers might not match a few assumptions, but these should be good.
Roger Pielke. Jr. February 25th, 2008 at 12:45 pm
OK, thanks. But do note that for AR4 the “future” started in 2000 (which is when they started their predictions).
These seem to be very small numbers, as a 1.0 C/century decrease is equivalent to a decrease of 0.1 in a decade, which seems to be a pretty small value. Looking forward to the details, Thanks!!
lucia February 25th, 2008 at 2:18 pm
Roger –
Yes. I’m aware that, when the IPCC wrote their document in 2007, the “future” begins in 2000. That’s obviously one of the truly bizarre things the IPCC does with regard to calling things “predictions” or “projections”.
Yes, in 2007, when they were “predicting” the temperatures from 2000-2020, the IPCC already knew the temperatures for 2000-2006 and part of 2007. So, of course, they were “right on” with those.
The math to figure out what would falsify can be done accounting for the already know data at the time the projection was made, but it’s stupid and tortured to do that.
lucia February 25th, 2008 at 2:45 pm
Roger–
I know.
The IPCC cheats by calling data that’s all ready in “predictions”. If we all put blinders on and pretend their 2007 document “predicted” temperatures in 2000-2006, then, of course, it starts to become next impossible to falsify.
A validation of 2000-2010 against data would involve using 70% data that were already incorporated into the “prediction”. One could, hypothetically come up with numbers that would, hypothetically, falsify this “prediction”, but the weather that has to occur in the last quarter of 2007, 2008, and 2009 is so statistically improbable that it can’t happen. Weather would need to cool very dramatically by an amount that is so far out of the normal weather variability bounds that, it is statistically nearly impossible. (Even counting this cold January.)
I don’t think I’m the only one who believes that post-validations should be limited to data collected after projections were made. Gavin did that when evaluating Hansen A,B and C. (I have differences of opinion about some things he did, but he did start evaluation with data that came in after the model was “frozen” and computations were underway.)
David Stockwell (34 comments.) February 25th, 2008 at 7:03 pm
Lucia, Nice post, I am looking forward to hearing our thinking on the construction of the graphs for falsifying the IPCC predictions. Question - why can’t the last ten years from the present be used? We know the IPCC have their fingers on the scale by including known data in predictions, but that is to their advantage. If nevertheless, there is zero trend for the last 10 years (which BTW there is) then why is that not falsification, despite the cheating. Alternatively, are there graphical lines that provide the same result as the ones you have shown, for the present back, rather than for the present forward?
Cheers
lucia February 25th, 2008 at 8:00 pm
David–
You have to be careful here. You can’t easily do statistics with endpoints, you need to do fits.
Here is a fit to the past ten years data:
If you do an ordinary least squares, the trend is positive; in fact the slope obtained this way is 1.8C/century.
What you’ve probably read others claim is that this trend is not statistically significant.
That’s true: if you limit yourself to only 10 years, and there is quite a bit of scatter, a measured trend of 1.8C/century is not statistically significant different from zero. But that’s not what I’m saying in my table. I’m saying to falsify the 2.0 C/century claim the trend itself coming out of the OLS has to be less than zero. It’s an odd coincidence that the break even point happens to be zero, but, oddly enough, it is.
Now, for why we need to test the accuracy of IPCC projections using data collected after their projections: this needs to be done to place the temptation for cherry picking beyond anyone’s reach. By saying we will stick with data starting after the IPCC published their report, neither you, nor I, nor the IPCC, nor anyone can cherry pick.
I don’t know what 2008 is going to end up as. The IPCC certainly didn’t know in 2007. And 2008 is the first full year after their report was issued. This is the year to pick.
David Stockwell (34 comments.) February 25th, 2008 at 8:35 pm
Lucia, thanks, great blog BTW, as a focus on verification statistics is very rich seam!
I recently looked at the trend on the monthly temperatures 10 years back using January 2008 HadCRUT data. The trend was almost zero.
Anyway, based on your table, there is a predicted trend that is falsified for every observed trend.
So, the the actual dataset and trend is not the issue for the moment. That was an aside. My question is about the constraints you are putting on that the new data must be from the present on.
I could imagine that each of these falsification scenarios have a different answer/graph.
1. Ten years on from the present.
2. Ten years back from the present.
3. Any ten years I get to cherry pick.
I don’t see how you can rule out 2 and 3 and admit only one. While I could imagine scenario 1 as being the strictest test, scenario 2 as requiring greater observed trend, and scenario 3 as having the most observed trend, I would think there is some quantitative relationship between them.
For example, I could imagine an analysis that says that this particular 10 set of observations has a 1% chance of being from that model prediction over that time period, even when the model covered the period in question.
lucia February 26th, 2008 at 4:43 am
David,
One problem with statistical reasoning is that for these numbers to really “work”, there should only be one experiment. If you start giving yourself choices (2)-(3), then all uncertainty intervals need to be shifted. That’s why we get so many arguments– no one can decide in advance which experiment is “right”.
In some sense, you can do 2-3 for something, but even if the IPCC says they are ‘predicting’ the past, they really aren’t. It’s the past. That’s why I exclude it.
Here’s why I don’t pick 2 or 3:
What you would be showing with your (2) and applying to 2001-2010 is that the IPCC ‘predictions’ don’t even describe the past. This is would certainly cause people to lack a lot of confidence in the IPCC, but the IPCC could say they always expect the rate to pick up, but it was lower in the early portions. Then where are you? The IPCC isn’t falsified because they say you started too soon. (Although, I think reading indicates they do expect the 2C/century in 2001.)
So, in this case, while IPCC ‘predictions’ failing this in 2001-2010 has some weight, it doesn’t quite falsify.
On (3)
You can’t just pick any set of years you like and apply this criterion. The statistics change if you get to pick whichever string you want. That’s because if you had 20 possible runs of years, we actually expect one of those runs will have the low slope by pure random chance. Obviously, you pick the low one for your case. The other guy picks the high one for his case and says it looks fine. And then where are you?
Now for (4)
If you always look at the most recent 10 years, you might at least be able to claim you see “the top”.
The difficulty with this strategy is that, if the real trend is 2C/century, you’ll “see the top” once every 20 years, but these will happens in strings. So, you’ll claim you are right for a few years, and with confidence, but you’ll be wrong.
The problem is,of course, that you are doing the equivalent of flipping coins over and over until you get the string you like so you can “prove” something about the coin.
Still, at least if you say you’ll always look at 10 years to find confirmation of “tops”, at any given time, you have only 1 experiment!
If it’s not a top… well… In a few years, you’ll be shown wrong. So, you will have erred.
As for using other data sets: Yes. Of course you can use Hadcrut. The standard deviations might be a bit different and this would shift things a little. I picked GISS because to do the calculation, I need a data set.
If Hadcrut and GISS results differ, well, we have a situation where one waits a year.
Can IPCC projections be falsified? Sample calculation | The Blackboard February 26th, 2008 at 9:49 am
[...] Comments: What weather would … [...]
lucia February 26th, 2008 at 12:51 pm
Hmmm… David, I looked at the Had-Crut data. Temperatures have been flatish since 2001, haven’t they?
Yeah, my normal inclination is to only look at data after predictions are made. But, if the IPCC’s basis is 2001-2010 for their claim, things aren’t looking good for their predicition even accounting for the fact that 7 years were already in!
Raven February 26th, 2008 at 6:22 pm
Lucia,
I read somewhere that HadCRUT is surface record that the IPCC uses in its projections so any falsification of the IPCC predictions should be done with that record.
GISS is typically higher than the other datasets because it includes estimates for the poles. However, these estimates are pretty dodgy given the poor coverage in the polar regions.
After reading your analysis I went looking to see if some of the more absurd predictions could already be declared falsified and found yet another twist that makes the warmer claims completely unfalsifiable: tipping points. According to warmer logic the earth will reach tipping points if the temperature reaches a certain level which will cause a rapid acceleration of warming. This means that they can always claim that catastrophe is ahead no matter what the temperatures do in the short term as long as there is some warming.
David Stockwell (34 comments.) February 26th, 2008 at 7:15 pm
Brushing up on Popperesque terminology, there are distinctions between unfalsifiable hypotheses (e.g. psuedo-sciences), and the strategy of using auxiliary hypothesis—hypothesis other than the test hypothesis which is assumed to be true and is needed to derive the test implication, and ad hoc hypothesis—an auxiliary hypothesis introduced for the sole purpose of saving a test hypothesis being threatened by adverse evidence.
I think ideas such as ‘tipping points’ and ‘heat in the pipeline’ could serve as ad hoc hypotheses, and that GW is falsifiable via data such as slightly increasing temperatures with increasing CO2. However, it is hard to see how they could be used to defend the theory against stable or declining temperatures. Tipping points is more of an auxiliary hypothesis to motivate the impacts side of AGW, i.e. to counter the argument that even if temps increase it will be at a rate we can adapt to.
I think that AGW proponents cant afford to let AGW theory be seen as unfalsifiable, but they will use every strategy to minimize ‘falsification risk’, such as using existing data to verify models, produce regular reports without evaluating past predictions, constantly modify the hypothesis, ‘move on’ etc. To governments, the IPCC endeavour could be seen as a form of avoiding ‘falsification risk’ by transferring the risk of failure to an external authority.
lucia February 26th, 2008 at 8:23 pm
@Raven– Obviously, no one can do a statistical test to falsify a prediction that is not quantified in the first place. Maybe tipping points exist. Or not.
As an empirical matter, only way we’ll ever know they do exist is if we reach them. The only way to falsify the claim is if a) someone tells us when the point will be reached, b) tell us how to recognize the tipping poing and c) what will happen afterwards. In that case, we could falsify the “when, how and what”. But otherwise… well, the tipping point may exist, but there is no way to disprove it with empirical data.
@David– I’ll be having a look. I’m actually rather new to the whole AGW game, so I don’t know what’s in all the documents. But, I do like to be a bit careful and not basically trick myself into cherry picking.
Part of the reason my answers can be a bit hesitant is I’m not all that familiar with the reams of data. My blog is something like a month and a half old. I started my open thread to hear the various pro/con arguments, and then I started blogging.
Anyway, it struck me that Roger’s question was one that can perfectly well be answered. The general question is common; to answer one has to make it more specific. But still, every experimentalist tries to figure how they would use data before doing an experiment. If you don’t think about how data would be used to perform a hypothesis test, how would you figure out how much data to take? Or which specific data? You don’t want to take tons of data and miss one thing that you really needed. And how do you figure out a sampling rate etc.?