Hypothesis test for 2C/century:
now with Monte Carlo!
now with Monte Carlo!
Sometimes, a picture is worth a thousand words.
See the blue vertical line at 2 C/century? That’s the central tendency for trends projected by the IPCC in the AR4.
See the bell-shaped curve? That’s a spread of trends due to “weather noise” that we might expect if the “weather noise” is similar to that seen on earth since 2001. (Assumptions made are described below.
See the orange vertical line hear 0.3C/century? That’s the lower ±95% confidence interval. Based on the assumed distribution of weather noise, we’d expect fewer than 2.5% of 90 month trends fit using OLS to fall below 0.3 C/century.
See the vertical green line at -0.4 C/century? That’s the trend based on merged NOAA/HadCrut/GISS Land-Ocean data.
Diagnosis: Based on this analysis 2 C/century, the central tendency for the trend projected by the IPCC in the AR4, falsifies.
Some details: The distribution shown above was generated by running a simulation of a process in which the “monthly averaged weather noise” is AR(1), with a lag 1 autocorrelation of ρmonth = 0.5044 and variability σ=0.1039 C. The trend was assumed to be 2C/century– which matches the central tendency projected by the IPCC in the AR4.
If that process describes the climate, and weather noise, the observed trend of -0.4 C/century is a result that would happen in fewer than 1 out of 40 events. So, the assumption that the underlying trend is 2C/century falsifies to a confidence level of 95%.
Discussion
That’s the major news. Rather than write an organized narrative, I’ll just answer questions I assume people will ask. Here they are:
Why use only three sources? I usually merge data from these five sources:
GISS, HadCrut, NOAA/NCDC, UAH/MSU and RSS. However, it seems many who doubt the 2C/century is falsified based on comparison to data since 2001 think I should stick to data collected on the surface. So, I eliminated the satellite data from today’s merge. (I will include the others in posts later this month.)
Why not use Gavin’s suggestion of weather noise based on models? Because
b) the properties of that weather noise falsify when tested against monthly data,
c) I think observed real earth weather noise should generally be given precedence over “model” weather and finally
d) I will never use “weather noise” that has been shown inconsistent with real weather noise.
Why these specific properties for “weather noise”? These properties reproduce the standard deviation and lag 1 for residuals to the Ordinary Least Squares fit for the observation consisting of a merged of NOAA, HadCrut and GISS from Jan 2001-June 2008. (These are lag1 correlation of ρOLS=0.458 and sT=0.102.)
What’s the conclusion based on the Monte-Carlo? The observed weather falls well outside the 95% confidence intervals for trends that might occur if the true trend is 2 C/century, while the weather variability is described by observations of weather since 2001.
How does this compare to Cochrane-Orcutt and OLS using the Tamino Recommended method for error bars? The 95% uncertainty intervals using Monte-Carlo were ±1.68 C/century. The 95% uncertainty intervals using Cochrane-Orcutt and OLS were ±1.60 C/century and 1.76 C/century respectively. So, when OLS applies, C-O rejects 2C/century at 95% confidence a bit too often and OLS with Tamino-pumped up error bars doesn’t reject often enough. The fraction of errors for each method is about equal near ρ =0.5.
What was the conclusion based on CO and OLS? Both methods rejected the 2C/century trend, just as for the Monte-Carlo.
Any caveats?
Sure.
- While assumption of red noise does account for larger amounts of weather noise at low frequencies than does white noise, there is a possibility of even greater proportions at low frequencies. If so, methods assuming Red Noise (aka AR(1) ) under-estimate the uncertainty intervals. It is also possible the Red Noise assumes too much energy at lower frequencies. If so, it will over-estimate the size of uncertainty intervals. That said, even these generalities could flip around depending on the specific distribution of the feature of the “weather noise”.
JohnV is concerned about the impact of the solar cycle, and I believe he’s planning to look at that.
- I simulated the monthly averaged data as AR(1). If the underlying weather noise is AR(1), the lag-1 correlations for the data will deviate from AR(1) somewhat. The precise effect on the computed uncertainty intervals not entirely obvious to me. I think using averaged data will tend to over-estimate the uncertainty intervals for longer trends, but I’m not sure. I plan to look at this next month when I examine red noise.
- I have reason to believe the measurements may contain some errors.
If so, the analysis must be extended to include the possibility of these errors. The precise effect on the uncertainty intervals is also not obvious to me. I’ll be doing that next month (it’s easy; so this is not just a plan.)
Are you wondering about the individual results? I’ll post those later on!
Comments
lucia (Comment#4235) July 17th, 2008 at 12:01 pm
Thanks Erik–
Bear in mind– Creating the graph does involve some assumptions about the data. But the assumptions don’t look too bad, and I’ll be doing a few things next month to overcome them.
Steve Geiger (Comment#4236) July 17th, 2008 at 12:25 pm
Thanks for the interesting post. I appreciate the way you write up your posts and, as in this case, include the ‘likely’ arguments that will arise. I really wish there was more of this type of exchange thoughout climate blogdom.
George Tobin (Comment#4237) July 17th, 2008 at 12:34 pm
A real analytical tour de force!! Thank you.
It looks as if the IPCC are increasingly in the position of being right only if they also assert that the chaotic swings that characterize reality are large enough to completely conceal that which they say is really happening. We are right, but you will never be able to tell, so there!!
I still think they will have to opt for bar chart graphs in 50-year blocks instead of line graphs to lucia-proof future projections.
The Terminator:
The SkyNet funding bill is passed. The system goes online on August 4th, 1997. Human decisions are removed from strategic defense. SkyNet begins to learn at a geometric rate. It becomes self-aware at 2:14am Eastern time, August 29th. In a panic, they try to pull the plug.
Sarah Connor:
And, Skynet fights back.
What year do the GSMs become self-aware and start looking for lucia…
Joseph (Comment#4239) July 17th, 2008 at 1:31 pm
What if it’s not weather noise in the conventional sense? Basically, what if the effects of weather noise you see in the historic record are not applicable to the last 10 years?
I actually have some data that *may* back this up, and I’ll write about that on some other occasion, but here’s the hypothesis.
It’s warmer than it should normally be given the long term temperature change trend (which I think is roughly 2C/century). When there’s imbalance, I’d suggest that local variability cannot cause the imbalance to be resolved any faster or slower (assuming the local variability ends before equilibrium is reached). Therefore, the trend gets corrected sooner or later. The way it’s corrected in this case is by a real reduction of the temperature change rate, which will last until we’re back on track.
Julian Flood (Comment#4245) July 17th, 2008 at 3:31 pm
Re comment 4329
Yes, but…
If you look at the raw SST data from Hadcrut3 (without the dubious Folland and Parker correction), you’ll see two periods of warming, 1910 to 1940 and 1965 to 2000. The rate is .14 deg/decade. If Joseph is correct then a simple ruler and pencil will indicate when warming will recommence.
Searching for “Original Caption: Folland and Parker [1995] Figure 3. Annual anomalies from a 1951-80 average of uncorrected SST (solid) and corrected NMAT (dashed) for (a) northern (b) southern hemisphere, 1856-1992. Only collocated 5 deg. x 5 deg. SST and NMAT values were used.” should find the graph.
Drawing a line and extending it to give rate of .14deg/decade might have saved a lot of money spent on climate models — I wonder if it falsifies? More to the point, what might be the mechanism? After all, the consensus theory has an underlying theory, more CO2 = more warming. Wheels, bus…
Speaking of wheels, there was a recent paper which made an effort to simplify the temperature record by chopping out the ENSO effects. That’s like chopping off a wheel from a car and declaring it a three-wheeler.
JF
JohnV (Comment#4271) July 18th, 2008 at 11:27 am
lucia,
I’ve been working on my Matlab code for doing this type of analysis. I’m running some tests before doing anything new, but I’m having a little trouble reproducing these results. I suspect the difference is in the way we define the AR1 weather noise. My code is supposed to be doing this:
Y(i) = m*X(i) + E(i)
E(0) = rand(0)
E(i) = rho*E(i-1) + rand(i)
where,
Y(i) is the temperature at sample i
X(i) is the time at sample i
m is the assumed underlying trend (m = 0.02C/year)
E(i) is the serially correlated weather noise at time i
rho is the lag-1 auto-correlation (rho = 0.5044)
rand(i) is a random variable with a normal distribution (mean=0, sd=0.1039)
Does that match what you’re doing?
I also have a quick question about your rho and sigma for the weather noise. I see two values for (rho, sigma) in the text:
(0.5044, 0.1039) and (0.458, 0.102). Which values were used?
Thanks.
lucia (Comment#4272) July 18th, 2008 at 11:59 am
John–
I use the higher set of values. When I run the OLS, it results in the lower values. This is a known result for fits to cases with finite values. When you use “N” data points, the expected value of the correlation is approximately
<r> = ρ + (1 + 4* ρ)/N + terms of order 1/N^2
The “N” is the number of data points. the ρ is the value you use to generate the data and the <r> is what you expect to get.
When I generate the random values, I wrote a code. I use:
E(i) = ρ E(i-1) + sqrt(1- ρ^2) u(i)
Where u(i) is gaussian with the standard deviation I need for the process.
If you take square both sides and take the expected value, you’ll see this gives the correct variance.
After creating the function, I ran it and checked that for zillions of numbers, I get the correct autocorrelation and variance. I also checked that my ‘us’ are normal.
With matlab, is rand normal? If it’s not, you need to transform.
lucia (Comment#4273) July 18th, 2008 at 12:18 pm
If you do this:
<E(i)E(i)> = rho^2*<E(i-1)E(i-1)> + <rand(i)rand(i)>
And recognize that you want the variance of E to be steady, <E(i)E(i)> =<E(i-1)E(i-1)> =<EE>
you’ll see your process ends up with
(1-rho^2) <EE>= <rand(i)rand(i)>
So, if your variance for <rand(i)rand(i)> = σ your variance for EE will be σ/(1-ρ^2)
I assumed that Gavin means we want the process where the variance for E is
σ. That’s why I have that extra term.
If I interpret it this way, I reproduce his 8 year variance for OLS trends.
JohnV (Comment#4275) July 18th, 2008 at 1:01 pm
Thanks lucia.
I’m actually using Octave’s rndnorm() function which returns a random variable from a normal distribution.
The sigma scaling seems correct to me. I’ve implemented it and can now reproduce your results.
Now that the tools are ready I can start looking at other things…
BTW, how do I get Greek letters in here?
Francois O (Comment#4276) July 18th, 2008 at 1:04 pm
Lucia,
This is great work as usual. The only question I have is about the use of that specific period. It’s not like we don’t have data for prior years. Of course there’s been little warming, or rather cooling, over the past 7 years, but what happens to all this if you take a longer period, say 1979-2008, or even 1959-2008 (so that it can be correlated, or not, with CO2 data)? I know you’ve discussed this in the past but I’m too lazy to go back to the earlier posts.
The precise question is: what is the trend and confidence intervals, using the same method, but taken over a longer period. Does that give us a narrower range of possible trends? Is there overlap between the range of trends you find, and that found over a longer period?
In the end, given that CO2 increases more or less exponentially, and that the assumed forcing is logarithmic, there should be a linear temperature trend associated with it, so it’s only fair to look for the linear component in temperature data. That’s not to say that that linear component is equal to the effect of CO2 (or GHG’s in their totality). Other possible forcings could contribute, positively or negatively, to that linear trend. You can end up doing like Scaffeta and West, and try to disentangle the possible forcings.
I think in the end, if we have enough empirical data on both temperatures and CO2, we should be able to put upper and lower bounds on the climate “sensivity” based on empirical observations alone, ie. not models. Ideally, one should use ocean heat content, though. We don’t have much data for this, but maybe your method could nevertheless give a similar range of sensitivities based on these data. All this would, in the end, help advance the debate, and our knowledge in general.
MarkR (Comment#4277) July 18th, 2008 at 1:18 pm
Lucia. Excellent. More questions. What is the the chance that the 8 year temperature observation correlates with the observed rise in CO2 for the period? Also, is nothing to be made of the apparent fact that the merged temperature record for the last eight years actually shows a decline? Is it really the case that the allegedly overwhelming force of CO2 in driving planetary temerature has in turn been temporarily overwhelmed for the last 8 years by some other currently mysterious, but more powerful force of nature? How are “Weather” and “Noise”, both presumably random, creating a discernable long term downward trend?
MarkR (Comment#4278) July 18th, 2008 at 1:20 pm
oops. I see Monckton has been addressing some of these issues: http://scienceandpublicpolicy......risis.html
lucia (Comment#4284) July 18th, 2008 at 2:47 pm
MarkR–
The 8 year flat trend is not inconsistent with warming. It’s just not consistent with 2C/century right now. Weather does exist, and does cause variations.
This flatness doesn’t disprove the theory that CO2 causes warming.
lucia (Comment#4286) July 18th, 2008 at 2:57 pm
JohnV–
Good.
BTW, although I convinced myself it doesn’t matter how I initiallize the red-noise for current purposes, I nevertheless initialize with white noise with the appropriate standard deviation. (If you subtract in the equation, it shouldn’t matter when generating data for curve fitting. But I decided: Why risk it? )
Looking at the 11 year solar would be good. I find I can only add one complication at a time, and I picked adding white noise first. Adding a periodic spike at different places was going to be later. (I was thinking add an “Enso Spike” at 4 years, and 11 year bulge etc.
The problem we will always have with the 11 year solar effect is that many people simply don’t believe it exists. Others, of course, believe violently in it. But, it’s definitely work looking at.
lucia (Comment#4288) July 18th, 2008 at 2:58 pm
Francois–
Excellent question.
First– yes. I can repeat the same thing for more years. I plan to. I anticipate getting qualitatively similar results to what I get with OLS or Cochrane-Orcutt. The uncertainty bands for the trends will generally all overlap.
I spoke to someone today. I’m thinking about ocean data.
John F. Pittman (Comment#4294) July 18th, 2008 at 5:12 pm
Lucia, I wonder if you can expand on MarkR’s comment “What is the the chance that the 8 year temperature observation correlates with the observed rise in CO2 for the period?” with respect to your weather noise as indicated in the actual weather noise you have been using. Your assumption(s) as is, etc. The question is that IPCC indicates an approximate value of 2.5C per doubling of CO2. The math is .2C per eight years. I would like to keep it at this valuse since this is the value that is responsible for the claim we need to mitigate rather than adapt. If you repeat your analysis substituting the predicted increase to temperature due to this assumption, and use your weather variation, can the 8 year trend be shown to invalidate the IPCC claim at 5%, 50%, or 95%? The answer is not trivial. The section on mitigation versus adaptation relies on this. If I remember correctly, the push for mitigation over adaptation depends on the result being 2.5+1.5, no -1.5. At 1C per doubling, adaptation would take precedence. We would have weather noise variation, the 0.2C per eight years tested against the actual fractional amount of doubling. I believe it was http://wmbriggs.com/ had a great post showing the linear trend of CO2 by month of year to get rid of oscillating wave.
Francois O (Comment#4296) July 18th, 2008 at 6:50 pm
Lucia,
“I spoke to someone today.”
Is that a rare event? Maybe the blog takes a little bit too much of your time?…
John F. Pittman (Comment#4302) July 19th, 2008 at 6:25 am
Lucia, using data from wmbriggs to estimate CO2, and IPCC 2.5+-1.5, I get that the 1Cx2CO2 sensitivity to end of June 08 is +0.058C, the 2.5C 2x is +.145C, the 4.5C 2x is +.232C, assuming that Jan 1 2000 is 0.0C. I have put it in monthly format. I use the average rather than the sinsoidal actual curve. To get .2C/decade one would have to use 2.87, rather than 2.5C for 2x, assuming a steady ln rise from 2000 to 2007 CO2 data. Perhaps .2C assumes an increasing ln rise, such that the average over the time span is .2C/decade.
lucia (Comment#4303) July 19th, 2008 at 7:14 am
John and Mark–
The difficulty with doing the statistical test you propose is that according to the IPCC, models, and even simple theories, the earth’s climate is not supposed to be in quasi-equilibrium. The temperature is supposed to lag the response to CO2. So, I can’t just compute the correlation for the past decade or so, show there isn’t any, and decree there is no connection. The response by those who believe there is a connection (which includes me) is to explain the theory associated with the phrase “It’s in the pipeline”.
John F. Pittman (Comment#4304) July 19th, 2008 at 7:27 am
Lucia, since the IPCC has claimed that the last part of the 20th century was due to climate forcing, just how long, and I would wonder what proof of such an extent, is this pipeline? As a self consistancy check, could you not assume that the stuff in the pipeline is approximately the same as the stuff we are recieving that was in the pipeline, and rule this pipeline stuff out as being a relevant factor?. If it is unquantifiable, or such a small difference that it can be essientally eliminated, and we are trying to quantify something, shouldn’t it be left out?
MarkR (Comment#4312) July 19th, 2008 at 2:25 pm
Does the current 8 year period of temperature rise correlate with any previous eight year period? What is the pipeline period specified by the IPCC, the Models, or any “Simple Theory”? Surely the “pipeline period should be subject to test and verification?
Roger Pielke. Jr. (Comment#4319) July 19th, 2008 at 5:40 pm
Lucia- Great stuff, clearly explained. You really out to write this up for publication.
Of course, another “framing” of your graph is that the observed (green line) is “consistent with” 0.2 deg C/decade since it falls somewhere within the bell curve . . . 
Ninety Month Trends: IPCC AR4 2C/Century still outside ±95% uncertainty bands. | The Blackboard (Pingback#4331) July 20th, 2008 at 6:47 am
[...] I do, and I think that makes the model… Hypothesis test for … [...]
jc (Comment#4532) July 29th, 2008 at 4:37 pm
Lucia,
I can’t even call myself a rank amateur at this stuff, so please pardon me if this is a stupid question. I was reading elsewhere about the concept of kurtosis and I tried to imagine how it would apply to your analysis.
Could you explain what would happen to your analysis if weather noise were distributed in a more extreme manner? Do you have any comments about why, or why not, you believe a more extreme distribution could be applicable to the discussion at hand? Also, were a more extreme distribution to have any validity, would that extend the length of time required to falsify within the confidence interval?
Thanks for your work. I am learning a lot from you.
lucia (Comment#4534) July 29th, 2008 at 5:41 pm
Jc– Asking about Kurtosis is not stupid. It’s actually rather sophisticated. I’ll write something up about that later this week. In fact, I’ll address the whole question.
As a general issue: The larger the variability, the more data we need to obtain a precise estimate of any mean variable. This includes the current trend.
jc (Comment#5053) August 14th, 2008 at 9:33 pm
Lucia, just a reminder (OK, I admit it — I’m a nag)
I know you have been really busy with more important things, but I am really looking forward to your discussion of kurtosis. If I missed it, please direct me to the right place.
Thanks again.
bender (Comment#5059) August 15th, 2008 at 9:46 am
How well known are the statistical properties of 1/f noise? This model is so universal it would be quite easy to justify in a published work. Gavin’s suggestion to use a GCM-based noise model is, umm, self-serving - to say the least. He readily admits that we know so little about the ocean’s state that we can not specify its state as part of the initial conditions of a typical model run. So they use random initial conditions. THAT is how ignorant we are about how the ocean functions. And THAT is the justification for a 1/f noise model.
1/f is going to blow those envelopes wide open and show that just about anything is consistent with the IPCC projection. i.e. You could not imagine a weaker hypothesis test.
Jeff (Comment#5099) August 17th, 2008 at 11:36 am
The period 2001-2008 was marked by a general decline in solar activity, which in the absence of other factors should lead to cooler temperatures. What was the motivation for picking that particular time period?
lucia (Comment#5100) August 17th, 2008 at 12:26 pm
Jeff–
The selelction of the time frame was discussed at length way back in February or so. But, basically, to test a “prediction/projection”, I wanted to use data that actually represents prediction, not a hindcast.
The AR4 was published in 2007, so in some sense, one might want to start then. But, various key documents were published in 2001. In particular, the TAR was published in 2001, making any the choice of any date prior to 2001 clearly a hindcast. Model runs prior to that would have been included in the TAR.
So, the choice was 2001. Jan 2001 just happened to be intermediate between a low in 2000 and a high in 2002. So, it gives somewhat intermediate slopes compared to other choices.
I do, from time to time, publish “bar and whiskers” graphs showing the sorts of difference we’d get if we picked other “first years”. The dates chosen are all based on publication dates of IPCC documents.
erik (Comment#4234) July 17th, 2008 at 11:41 am
Beautiful picture. I appreciate all you do, but this is by far the single best visualization of the “falsifiable warming?” data I have seen. The graph conveys a much better sense of the odds than just debating a couple likelihood numbers. Very nice.