The Blackboard

Thanks for posting this Lucia.

Question: will you post something on the applicability of AR(1) versus ARMA(1,1)? As you know, Grant Foster, that sensitive and thoughtful fellow Tamino, now claims that using AR(1) yields inaccurate (too narrow) confidence limits, and then claims measured trends do not call into question predictions of warming.

Where can we access the HadCRUT data now? There have been multiple adjustments made this year, so it’s been difficult to keep up.

I meant to ask is there an archived data set to track the changes.

SteveF—
I will eventually post something on that. But, I’m going to write a paper, including the discussion so I’d rather wait until I’ve trippled checked some stuff and checked for “robustness”. That said: AR(1) appear to give pretty darn good uncertainty intervals! (I even admit to being stunned how good. But more later.)

Lucia,

Can you give a hint of the paper’s title?

Lucia, the trend time is far too short to draw any meaningful conclusions from, they don’t have decadal resolution.

bugs–
Why do you think there is something important about decadal? Also, could you define “meaningful” in your second sentence? Because otherwise, you are statement is neither true or false; it’s just meaningless.

“an IPCC nominal projection of “about 2C/century””

Yet again, you misquote. No IPCC document contains the phrase “about 2C/century”, and yet you put it in quotes to imply that you are quoting verbatim. Why?

Hunter–
Thanks for noticing my typo. I keep forgetting they expressed it per decades. I edited to say “about 0.2 C/decade”. Of course, this makes no difference to the article or figure.

Lucia, I’m guessing that bugs is referring to the fact that many climate models fail to capture the physics that drive short-time fluctuations in their simulations.

Carrick– That may be what bugs is thinking… or not. If he were to say, then that could be discussed. I’m guessing his is just parroting something he heard somewhere, which he does not fully understand.

“bugs–
Why do you think there is something important about decadal?”

That is roughly the period of time you are testing over.

Bugs– Uhmm… Huh…. But you were complaining I don’t caputre “decadal” variations… but now you say I do test over roughly the period required?!

“lucia (Comment#28763) December 19th, 2009 at 8:13 pm

Bugs– Uhmm… Huh…. But you were complaining I don’t caputre “decadal” variations… but now you say I do test over roughly the period required?!”

I’m guessing you can’t. Approximations to El Nino, PDO, etc, happen in the models, as I understand it, but can’t predict when they will happen. Hadley is just trying to get that happening now, but as Trenberth famously said, we can’t explain the transfer of energy within the climate, so you may not see it turning up in the temperature record within the time span you are looking at. There is also the issue of the base you are starting from. You need to be using the long term trend as the model starting point, not a short term re-basing. As you have observed, the starting point makes a huge difference, because the signal is drowned out over the short term by the noise.

Bugs– No you can’t predict when El Nino, PDO etc. will happen. But that doesn’t mean we can’t test projections using statistical models. Those statistical models are precisely because we can’t predict.

The only thing about short term comparisons is the noise is larger. But if the difference between projections and the weather is large enough, you can say something. That’s why I want you to define what you mean by “any meaningful”.

I think being able to detect the multi-model mean in the AR4 is too high is “meaningful”. You may not.

Lucia,

Thanks for an insightful statistical look at the HadCRUT data over the last decade.

I just wish the IPCC would better utilize statisticians in their research and preparation of their reports. The significance of their conclusions are indeterminant without appropriate statistical analysis.

I agree with bugs sort of.

“There is also the issue of the base you are starting from. You need to be using the long term trend as the model starting point, not a short term re-basing. As you have observed, the starting point makes a huge difference, because the signal is drowned out over the short term by the noise.”

The models started at 2001 so if you want to asses their skill you have to start there. If bugs thinks we ought to look at long term trends, say 30 years, then I’d be quite happy to wait until 2031 and then asses the models. But I’m also concerned about LTP, processes with time scales on the century level. If at one end we are concerned about noise at the other end we need to be concerned about LTP. So lets wait a century. No wait, I’m concerned about 1000 year processes.

And what are these 1,000 year processes? That Koutsiannis paper is the biggest mess since Gerlich and Tueschner set pen to paper.

SteveMosher–
As you know, I have posted longer term trends on many occasions. The models reject at between 90% and 95% for all start years since about 1960.

But I’m still not sure what precisely it is that is bothering bugs. Also, I know he reads this blog often enough to be aware that I’ve done long term trends, but happen to highlight shorter term tremds. So… I really don’t get what it is that bugs bugs.

steven mosher

“But I’m also concerned about LTP, processes with time scales on the century level.”

Ah, you mean like those extrapolations that assume this whole century we will be able to burn as much fossile fuel as we would like? That was the first thing that stunned me about the IPCC projections: The sheer hubris to think that we are actually able to reach a peak of, say, 800 ppm CO2. It is, to say the least, questionable to think that will be reached.

bugs:

That Koutsiannis paper is the biggest mess since Gerlich and Tueschner set pen to paper.

bugs you are such a poser.

bugs (Comment#28789) December 20th, 2009 at 6:16 am
And what are these 1,000 year processes? That Koutsiannis paper is the biggest mess since Gerlich and Tueschner set pen to paper.

Bugs. You know when it comes to explaining “divergence” climate scientists have no trouble throwing up all sorts of possible explanations.. could be this could be that. dont make me quote these speculations. You’ll just look silly. So, likewise, I feel no shame in speculating that there might be LTPs, especially in a chaotic process.

carrick, I think bugs should debate Tom Vonk.

bugs should also read a stats primer, he could then understand how short term data means greater error and so harder falsification — if we had his 30 year data the eroors bars may well be narrower and the falsification clearer. Also if Lucia used a models from 30 years ago bugs would say use the modern models — same old “hide the decline”. And whenever Aritic ice is discussed bugs highlights the decline 2000-2007 — which is of couse less than a decade, same double standard

Andrew Kennett (Comment#28861) December 20th, 2009 at 7:50 pm

bugs should ….long strawman

No.

Hi Lucia,
As an engineer with a lifetime of experience in electrical measurement I cannot understand that in all the discussion I
have not seen the words Quality Assurance mentioned. All of our measurements and instruments had to have traceability and customers would full access to inspect our labs and our measurements. Every engineering firm dealing with Government or Semi-Government had to have similar QA procedures. Now while I can understand
that it would be difficult to apply QA to scientific theory it appears to me that the measurement of temperature should definitely have QA procedures and documentation. These should be available on request from the authorities concerned as no Government purchasing body in Australia or other Western countries will accept product from firms without QA. It is totally inconcievable to me that Anthony Watts and his crew of amateurs should be required to perform basic site QA .
The fact that not only do we have measurement without any quality assurance, the very processes used are not divulged.
I think all the climate warmists need is a detailed independent Quality Audit and global warming will be greatly reduced!

bugs don’t confuse the Strawman Fallacy with the Wizard of Oz it does you no merit

@Bugs

We don’t have enough reliable temperature data to answer the question of what 1,000 year process there might be, but being a little familiar with chaotic systems and fractal patterns (spatially and with temporal analogues), I would be highly surprised if there weren’t millennial or decamillennial scale components to climate patterns as well.

I recognize there are limits to the scale of such chaotic organization, but to my knowledge we’ve only found them on the cosmological level.

@Peeke

While there are definitely constraining factors on the availability of carbon to input enough CO2 to reach 800 ppm, too many unknowns remain. I suspect peak oil has actually arrived, but if it has, it happened much later than original predictions, largely because of advances in technologies to both discover and gain access to oil. While the current global economy means asset and commodity deflation, it’s likely that oil shale will eventually become economical to use, though, again, later than predicted.

The other big mitigating factor I see to hitting 800 ppm is biosphere response/feedback to increasing CO2 concentrations. The more there is, the more plants will absorb until a new equilibrium point is reached.

Quite frankly, we’ll need both increased plant density and temperatures if we’re ever going to have cloned apatosauruses and their predators thrive anywhere but Costa Rica.

I posted the following qustions over at Tamino´s “How long” thread but unfortunately they didn´t pass moderation. I was genuinely interested in understanding how his analysis ties in with the *short-term* climate sections of the IPCC and what one can interpret from trends that approach the minimum significant period he posits (15 years) but clearly diverge from the IPCC short-term projections. It was Tamino´s view I was interested in but If anyone would care to comment, I´d be obliged.

“Two questions spring to mind:

1) When the IPCC itself makes predictions of decadal trends (0.2°C per decade over the
next two decades) in their “Understanding Near-Term Climate Change” chapter of the WWI TS http://ipcc-wg1.ucar.edu/wg1/R.....int_TS.pdf , are they also making statistically untenable statements?

2) Assuming that indeed the ARMA (1,1) is the best error model and hence 15 years is the minimum period to calculate significant trends, would there be nothing to conclude regarding the above prediction/projection if the OLS trend of the last 10 years had been, say, -0.3C?

Best regards,
Mikel “

Hello Lucia,

It seems to me that when you look at the climatological history 2001-present that is it not at all surprising that the IPCC mean model prediction will have gotten it wrong. We went from the peak of a moderately strong solar cycle to the deepest solar minimum in approximately 100 years. And, the first several years of the period were dominated by El Nino, while later years were dominated by La Nina.

The IPCC says that you need several decades of data to detect and attribute climate change, because of such variations mentioned above. The IPCC identifies a .1 deg C oscillation in temperature associated with the normal solar cycle. Since the current minimum was significantly deeper than a normal solar minimum it stands to reason that the cooling influence of the shift from maximum to extreme minimum will have caused a temperature cooling > .1 deg C.

Combine that with the cooling influence of ENSO over the period in question and I think it’s apparent that an underlying trend of .2 deg C/decade will have been offset quite nicely by unpredictable natural variation.

With a moderately strong El Nino likely to continue through spring, and with Solar Cycle 24 showing signs of finally picking up, Hansen is predicting a 50+% chance of a new record in 2010. We’ll see.

I’m no statistician, but I suspect, and you can clarify for me if I’m wrong, that your statistical test doesn’t capture the slower moving ‘noise’ of the solar cycle?

Benjamin–
Would it surprise you to learn the models also fail when compared since 1980? Also why do you think ENSO has a cooling effect since 2001. Also, note that the model is rejecting from 2000– that was closer to a La Nina.

bugs asks:
“And what are these 1,000 year processes?”
.
Umm, ocean circulation? Still didn’t read Brooker yet? Wunsch? Notably, Mann’s latest is starting to encroach on his hypothesis: a 1500-year cycle in the THC, last peaking during the MWP. Mann’s back-pedaling. Still using “MCA” terminology though.

Lucia wrote: “Would it surprise you to learn the models also fail when compared since 1980? ”

Somewhat. I’d be interested to see your analysis. The time series 1980 to present is still going to be influenced from the effects measuring from the peak of a large solar cycle to a once in a century solar minimum. If your conclusions would be changed by that ~.1 deg effect then you might want to include it in your analysis.

And: “Also why do you think ENSO has a cooling effect since 2001.”

For instance the SOI has a positive trend 2001-present, which is towards La Nina: cooling. That effect will minimized as we progress into El Nino, but as the time series stands at present, El Nino, and sub-El Nino but still warm conditions dominated the early years of the series, and La Nina the later ones.

I just think the once in a century solar minimum has to be having some measurable effect. The IPCC says you have to watch out for that variance over the course of the normal cycle, much less a once in a century low.

Benjamin
You can see what you would get applying a similar analysis to the one in this blog post but selecting different start dates here:
http://rankexploits.com/musing.....servations.

That graph is in march, so things are shifting down a little, but not all that much.

With respect to the solar influence– there are two issues that are worth keeping separate:
1) Is the observed trend consistent with the multi-model mean, which was obtained based on the what the modelers chose to incorporate as forcing and
2) If it’s not consistent, is the problem that the modelers can’t predict forcings? Or that the problem is with the physics in the model.

A t-test cannot answer the second. It can only answer the first. So, if for some reason, the modelers were unable to account for the solar variability properly, then they could be the explanation for the rejection, but it wouldn’t make the rejection any less real.

For instance the SOI has a positive trend 2001-present, which is towards La Nina: cooling.

I don’t know where you get your information. This is the graphic of the ONI in NOAA’s enso update posted today. I added a line to indicate 2001, which was neutral. 2001 was La Nina.

Note Hadley is currently rejecting if we start in 2000 or 2001. (In March, with Hadley it was rejecting for start years in 1997, 1998, 1999, 2000,2001,2002, 2003, 2004. I prefer tests after the SRES were frozen, and also after the last major volcanic eruption because it is very easy to show that this method will understimate the d* if there are known non-linearities in the multi-model mean– as after volcanic eruptions.)

Anyway: might the issue be the sun? Well… maybe. But it’s not as if the models all stuck the solar constant at “high” and then rolled forward into the future. So, there is no particular reason to think the solar issue is 0.1C.

Lucia:

I came across your chart above in an on-line discussion. Could you clarify for me some things in the plot? What exactly do the lines on your plot represent?

1. The purple line (this seems to be a 0.2 degC/decade increase… is this correct?)

2. The declining solid red line.

3. The inclining dotted red line.

4. The declining dotted red line.

5. Why do all the lines intersect in the middle of the plot?

Thanks.

bugs,

In addition to the millenial time scale of deep ocean equilibration, a quasi-periodic climate cycle with an average period of 1,500 years has been proposed to explain both Dansgaard-Oeschger and Bond events. A proposed driver is a resonance between the ~90 year Gleissberg and ~200 year de Vries solar cycles.

Lucia wrote: ” [Ben wrote:] “For instance the SOI has a positive trend 2001-present, which is towards La Nina: cooling.”

I donâ��t know where you get your information. This is the graphic of the ONI in NOAAâ��s enso update posted today. I added a line to indicate 2001, which was neutral. 2001 was La Nina.”

Do a regression on the ONI (or other ENSO measurement) 2001 to present, you will find a negative slope, which is cooling. For the ONI, I get -.07/year. Over 9 years the change in points on the trend would be -.6, which is the magnitude of going from neutral to a weak La Nina.

ONI data here http://www.esrl.noaa.gov/psd/d.....n/oni.data and here (where there is more ’09 data posted) http://www.cpc.ncep.noaa.gov/p.....ears.shtml

SOI data that I was using before: http://www.cpc.noaa.gov/data/indices/soi

Lucia also wrote: “Anyway: might the issue be the sun? Wellâ�¦ maybe. But itâ��s not as if the models all stuck the solar constant at â��highâ�� and then rolled forward into the future. So, there is no particular reason to think the solar issue is 0.1C.”

There is reason to think it is of that magnitude. The IPCC figure of .1C oscillation is for a normal solar cycle during which cooling from the minimum is cut off by the rising new cycle. Hansen has estimated that cooling would be roughly .2C if the period were slower so that the system could approach equilibrium. Normally we would be a couple of years into the rising solar cycle by now, so it’s logical to figure that the depression from peak to current is > .1 less than .2, let’s say .15 with a wide error bar.

The models presumably were in a an average state of average solar input at the time of the start of the projection (am I correct? I think that they are not tuned to present weather conditions before starting) and they would project, on average, an average solar state going forward, I presume. That condition would be a neutral effect on climate from solar for the IPCC mean model prediction. In reality we have 2001 at the peak of a moderately strong solar cycle and now we are well past a normal solar minimum. So the magnitude of cooling effect 2001 to present is likely in the ballpark of .15C. And the the cooling effect 1980 to present is also likely in that ballpark, because 1980 was the peak of a very large solar maximum.

It’s an unusual solar event we’ve witnessed. But, there are signs of life, with multiple sunspots on the sun at present.

http://www.spaceweather.com/index.html

http://www.spaceweather.com/im.....51srgdi3q4

The purple line (this seems to be a 0.2 degC/decade increase… is this correct?)

Yes. This is the nominal rate of increase discussed in the AR4. However, the intercept from the AR4 would be different. I sometimes show the actual curves from model-data.

2. The declining solid red line.

3. The inclining dotted red line.
That is the best fit trend passing through the data over the time period indicated. If you examine the formula for best fit trends, the line always passes through a point that is the (average of all X’s, average of all Y’s).

4. The declining dotted red line. Same as above but it’s the +95% confidence interval.

5. Why do all the lines intersect in the middle of the plot? Because I only want to compare the trends. So, I all pass through (average of all X’s, average of all Y’s). I could add the uncertainty in the intercept and the red dotted lines would make two “Vs” separated by a bit. But my purpose is merely to illustrate how the nominal trend compares to the upper and lower possible trends.

Thanks.
Your welcome.

Benjamin–
Of course El Nino is a factor. But

a) Why would you pick an index that ends in January when the post discusses trends through Nov? Also, since the post also shows 2000-now, why not examine 2000-now? The trend in MEI from Jan 2000 is -0.001 which is very slightly negative — but almost as neutral as possible. (In contrast, the trend since Jan 1999 is 0.042, which is 42 times as large.)

Since MEI is one of the factors in the “noise”, this doesn’t explain why the trend is outside the uncertainty intervals If you want, go ahead and find the paper that correct for MEI, and do that. You’ll find it makes the uncertainty intervals smaller. That very small trend is

There is reason to think it is of that magnitude. The IPCC figure of .1C oscillation is for a normal solar cycle during which cooling from the minimum is cut off by the rising new cycle.

But you are assuming the models got to the top of the solar cycle around 2000 and stayed constant. That is not so. Some of this 0.1C from the top to bottom of the cycle should already be reflected in the model forcing choices. Also, you might want to look up your Hansen quote. Prior to the slow warming, he pretty much said this was the upper possible bound.

he models presumably were in a an average state of average solar input at the time of the start of the projection (am I correct? I think that they are not tuned to present weather conditions before starting)

You are partly correct and partly incorrect. Different models did different things based on their judgement of what was both necessary and important. Some kept an 11 year cycle, some kept the cycle through 2004 (because that’s when the last group kicked in the SRES rather than historica forcings), some dropped the solar from high around 2001 to medium right away. (This would likely have the effect of making the model trends be low, because the earth’s climate system is slow to respond and it means there is less overburden of warming relative to what occurred during the period.)

So, the sun may be an issue– or not. But that is separate from whether or not the projections are inconsistent. Given the totality of what they did, there is no particular reason to expect a 0.1C difference.

“Why would you pick an index that ends in January when the post discusses trends through Nov?”

I did not, the second link I gave shows through October ’09, and the trend I calculated was based on the data through October. Also, you were the first one to bring up the ONI, I originally discussed the SOI. Any of the main ENSO proxies will give the same result.

I picked 2001 because that is your ‘default’ start date. I agree that a start date of 2000 provides an essentially flat trendline in ENSO. So, I presume a roughly flat influence on global temperature. As such a 2000 start date eliminates the issue that a 2001 start date has.

Regarding solar, so, they do use historical solar forcings up to 2001, or 2004 as the case may be? So, they start from high and either go to average, or simulate an 11 year cycle? (Sorry for the questions, I haven’t read that chapter in the IPCC in quite a while). Any of these strategies will overestimate the solar forcing 2001-present or 2004 to present, because neither one comes close to simulating the once in a century low in solar input that we are experiencing.

A 2001 drop to average would underestimate the forcing in the first few years, but that is over weighed by the overestimate over the last 6 years or so. A 2004 drop to average, well 2004 was already down to roughly average, so 2004 to present it overestimates solar.

Any simulation of the 11 year cycle would overestimate solar input the last 2 years when we would normally be climbing back into the new cycle. Models using a simulation would be the closest to reality, but would definitely not capture the full extent of the decline.

“Given the totality of what they did, there is no particular reason to expect a 0.1C difference.”

It’s an interesting question. You’d have to analyze how the different models handled it to see how big a deviation from actual solar forcing there was.

The way to get around the worst of the effect would be to pick a start date in a solar minimum, i.e. 1996, but then you get the huge El Nino of 97-98 which will cause a cooling trend in ENSO measurements ’96 to present, IINM.

All of these kinds of issues are why the IPCC says you need several decades of data for detection and attribution studies. In the last 15 years we’ve experienced some very notable ‘weather’ that definitely influences trends: the El Nino of the century in 1998, and a once in a century solar minimum which is only now possibly coming to an end.

“Also, you might want to look up your Hansen quote. Prior to the slow warming, he pretty much said this was the upper possible bound. ”

It was in the context of discussing what kind of effect a Maunder Minimum type solar event would have on global temperature, a couple of years ago. At any rate it is evident that if the ‘normal’ cycle produces a .1 deg oscillation as discussed in the IPCC, then an abnormally low and long minimum would produce a deeper dip at the low side of the oscillation.

Benjamin–

I picked 2001 because that is your ‘default’ start date. I agree that a start date of 2000 provides an essentially flat trendline in ENSO. So, I presume a roughly flat influence on global temperature. As such a 2000 start date eliminates the issue that a 2001 start date has.

Yes. I picked it based on publication date of the SRES; 2000 predates that.

Picking 2000 doesn’t avoid issues. It can avoid it temporarily but only in a way that forces you to move your start date around. We’re moving into ENSO. Should I constantly change start dates to get a flat trend on various ENSO metrics? At the top of El Nino, should I switch to a start date of 2002 reasoning that doing something else isn’t fair?

The variability of ENSO contributes to the estimate of the uncertainty. So, I basically rely on that.

Regarding solar, so, they do use historical solar forcings up to 2001, or 2004 as the case may be?

Depends. Chad downloaded some of the solar choices. Prior to 2000 different modeling groups picked different solar records. After 2000, some modeling groups set solar to constant, when they kept it constant, most did a step change down. Some kept it at an 11 year cycle around the mean during the 90s etc.
Presumably, they all did what they judged best in some regard.

The way to get around the worst of the effect would be to pick a start date in a solar minimum, i.e. 1996, but then you get the huge El Nino of 97-98 which will cause a cooling trend in ENSO measurements ‘96 to present, IINM.

I just picked 2001 based on publication dates. Do the test…watch then wait.

At any rate it is evident that if the ‘normal’ cycle produces a .1 deg oscillation as discussed in the IPCC, then an abnormally low and long minimum would produce a deeper dip at the low side of the oscillation.

BTW: Have you read the solar paper this comes from? The method of detecting that 0.1C seem to be like reading t-leaves. It’s got so much filtering and sifting going on one has to wonder. Many people seek this solar signal. Most can’t find it.

Model E has quite a few sensitivity runs available. Some visitors have looked at models to try to detect the solar signal. It’s much weaker than 0.1C in ModelE. So, is it 0.1C over the cycle? I don’t know. If it is, we are due to come out of this minimum soon, and we’ll see, right?

By the way: If you are correct that recent weather has been more variable than usual, then… well.. that means the uncertainty intervals could be larger than they should be!!!

“By the way: If you are correct that recent weather has been more variable than usual, then… well.. that means the uncertainty intervals could be larger than they should be!!!”

I return to a question/comment that I made in my first post. Would your statistical test capture the slow ‘noise’ of a constant downward trend in solar input that we’ve seen 2001-present?

ENSO, there are several of those cycles in that time period, so I would expect their variation to be accounted for in your test, but the almost uniform down-trend in solar since then?

Benjamin–

but the almost uniform down-trend in solar since then?

Why do you think it’s a uniform downward trend? Do you mean monotonically decreasing? (If you’ve seen charts of solar intensity, not even clear it’s that!)

If the anticipated response to the variation in solar is expected to be perfectly linear during the analysis period, it won’t contribute to the residuals and it wouldn’t contribute to the residuals. If it’s non-linear, and shows correlation, the variation contributes to the residuals and to the estimate of the lag-1 autocorrelation and so add to the uncertainty intervals.

Will it contribute the right amount? Probably not. But it contributes some.

You tell me what you think the expected value of the variation in solar constant is since 2001, and also go look up how it was included in all the models and we can try estimate this effect.

But this is why I say it could be caused by solar– or not. We don’t know if the peak to trough is 0.1C– that’s not agreed on. (It’s cited by the IPCC, but this is not, by any means, a robust finding.) Even if it is 0.1C, that doesn’t mean the models should lag 0.1C because various models accounted for this in different ways. (Some not at all, some with an 11 year cycle etc.)

So… we don’t know!

Thank you for the dialogue!
B.

Lucia,

In that chart you state that “the IPCC projection… lies outside the 95% uncertainy intervals, although just barely”

A mark 1 eyeball would suggest it is a more than a another standard deviation outside – possibly pusnig it below 0.5% likelihood territory.

I don’t think I would characterise that as “just barely” outside the 95% uncertainty intervals.

Of course, to be pedantic I meant a “standard error”