The Blackboard

Where Climate Talk Gets Hot!

Skip to: Content | Sidebar | Footer

Adding Apples and Oranges to Cherries.

13 October, 2009 (10:57) | Climate models Written by: lucia

Recently, Chip Knappenberger commented on some cherry picking by both Richard Lindzen and the fellows over at RC. Roger commented. I’m posting to add my two cents, focusing on two points:

  1. Observing that RC may have mixed an apples to oranges comparisons into their cherry pie.
  2. Evaluating a suggestion by Gavin’s that appeared in a comment at RC. In that comment, Gavin appears to suggest that Chip failed to accounted for lag-1 autocorrelation when estimating his uncertainty intervals. As far as I can tell, it appears Gavin guessed incorrectly and Chip accounted for the lag-1 autocorrelation when computing his uncertainties.

Chip’s article

Chip Knappenberger’s article focused on how both Stefan Ramstrof of RC and Lindzen’s precise choice of start year and observational metric affects the points they wish to advance. RC posted an article where they highlighted 10 and 11 year trends based on GISS only, happen to show warming, while Lindzen focuses on a 15 year trend, which he claims does not show statistically significant warming. I’m not going to summarize Chip’s article, as it’s already clear, and you can read it here. Instead, I’ll simply show his graph:


Chip’s graph shows trends computed beginning in September of the indicated start year and ending in August 2009. The horizontal red line at 0.2C/decade is the nominal trend the IPCC projected based on model simulations during the first part of the 20th century.

This trend of 0.2C/decade is, in fact, the value Stefan refers to in the conclusion to his post at RC which discussing temperatures back to 1980 closes with the statement, “The bottom line is: the observed warming over the last decade is 100% consistent with the expected anthropogenic warming trend of 0.2 ºC per decade, superimposed with short-term natural variability.”

So, when specifically commenting on Stefan’s post, it was reasonable for Chip to include this line when discussing the cherry picking aspects. In truth, if you accept the “about 0.2 C/decades” as the point of comparison for any and all start years back to 1980, a cherry picker might pick 1991 or 1992 as a start year for comparison and say:

Or on the other side of the coin:

• Global warming did not ‘stop’ 10 years ago, in fact, it was pretty close to model projections. [Using the GISS and NCDC records beginning in 1998 and 1999]

Global warming is proceeding faster than expected. [Using the GISS record staring in 1991 or 1992—the cool years just after the volcanic eruption of Mt. Pinatubo]

But you know what?

I think to justify the latter, italicized claim, the person who made it would have to do three things. These are a) cherry pick using GISS only b) cherry pick starting 1991-1992 (for the reason Chip stated) and c) make an apples to oranges comparison.

How do apples to oranges sneak into the mix?

These other fruits sneak in because, not withstanding Stefan’s conclusion that suggests 0.2C/decades is a fair comparison (which it sometimes is), the model simulations don’t predict 2C/decades for the trends that happen to be computed starting specifically in 1991-1992. Moreover, under projections based on the A1B scenarios, the trend of 0.2C/decades is always lower than the multi-model mean for trends beginning in years starting in 1980 through 2005 and ending in the most recent month.

(Note: In fairness to Stefan, he did not make a specific comparison of observed trends starting in 1991 or 2 to “about 2 C/century”. But this is just what some might do if they were picking various years and not computing the model trends for those years. FWIW, this has been done at some blogs. )

Let’s add the multi-model mean to Chip’s graph.

Let’s examine what happens if we add the multi-model mean computed from 22 AR4 models forced with the A1B scenario to the graph. Below I’ve added the multi-model mean and temperatures from Hadley:
NoUncertainty
Notice that, as in Chip’s graph, the observed trend starting in 1991-1992 exceeds 0.2C/decade. So, if the models had “predicted/projected” that trend, one might conclude that for those choices of start year, the models under predicted warming. Then, one might say “Oh. For some years, the models over predict and for some they under predict.”

However, recall that roughly 2/3rds of the models used in the AR4 were driven by volcanic aerosols. Simulations from those models also contain a dip in temperature due to Pinatubo.

Consequently, if we compute the trends starting near the temperature drop due to Pinatubo, and compute the trend for the models and the observations on the same basis, the models still overpredict the trend, and by roughly the same amount as for nearby start years.

In this case, we modify Chip’s possible conclusion to, “Warming is going more slowly than projected when computed for all start years since 1960.”

So, currently, when you are reading comparisons, it’s useful to watch out for the “apples to oranges” mixed in with the cherries. Because right now making the models look good requires a fruit-compote effect.

GISS with ±90% uncertainty intervals.

I’ll now turn to the issue of the uncertainty intervals, which is characterized by Andrewt in comments at Rogers as follows:

Gavin is pointing out that the article has contains a statistical error and the significance tests depicted in the diagram at top of your post are incorrect. Perhaps you should update your article so that (more) readers aren’t mislead.

Andrewt didn’t provide any additional information, but it appears he is referring to this comment after Stefan’s post

Well here is an intersting post that suggests that the main post here is based on cherry-picked data. http://masterresource.org/?p=5240 . It summarizes in an easy to understand manner the data for all major temperature indexes and their trends for a a variety of periods over the last twenty years or so.

[Response: Vaguely ok, but he has failed to take the clear auto-correlation in the monthly data series into account and so his statements about significance are all biased to be over-definitive. - gavin]

Before proceeding, bear in mind two things: 1) The RC article associated with this comment does not discuss uncertainty at all and 2) If Chip made this error (and it’s not clear he did) then the conclusion that might be modified is Richard Lindzen’s: We might discover it’s true that there is no statistically significant warming over the past 15 years!

So, to explore the issue of uncertainty intervals, I computed the ±90% uncertainty intervals correcting for “red noise” using the method discussed in Santer 2008 (which lists Gavin as a co-author) and added these to the figure above– removing Hadley temperatures to minimize clutter:

GISS

Comparing the lower blue ±90% uncertainty interval to the 0C/decades, I found the trends computed with start years after 2001 are negative, but this is not statistically significant at p=10%. The trend computed starting with start years of 1997 and earlier are positive, and this is statistically significant. Between these two, the trends are positive, but they pop in and out of statistical significance at p=10%. (I get comparable different results for p=5%.)

My start times are January; Chip’s are September. But his open and closed circles indicate he and I are getting comparable results when assessing which trends are statistically significant. Had he not corrected for red noise, his results would look very different.

So, it appears Gavin may have been mistaken in his guess and Chip did correct for red noise. (Of course, we could just ask Chip. But, even we’d then check anyway, right? :) )

Hadley with ±90% uncertainty intervals.

Now, let’s repeat the above using Hadley:
Figure N: Multi-Model Mean trends compared to Hadley.
For this case, the trends at p=10%, the trends are not statistically significantly different from 0 unless computed from the start year of 1995 and all earlier years. (If we cranked this up to p=5%, the we would need to start even earlier, the trend since 1995 is not statistically significant, but the trend from 1996 is statistically significant.)

Though (for my convenience) I start my trends in January, while Chip starts his in August, my results for which cases are statistically significant are in rough agreement with Chips graph and narrative, so it appears Chip did account for the lag-1 autocorrelation when estimating his uncertainty intervals. Either Gavin guessed incorrectly or Gavin prefers a statistical model that provides wide error bars than those that would be obtained using the method discussed in Santer 2008. Chip also appears to be correct when saying Lindzen’s claim of no statistically significant warming is in error.

Summary

For those who lost track, the summary is:

  1. It looks like Chip’s article did models big favor by suggesting a cherry pickier could compare trend starting near the eruption of Pinatubo to “about 0.2 C/decade” and then decides models might underpredict warming. In fact, models predict more than “about 2 C/decade” warming from those start times. The observed warming from those start dates is less than indicated by the multi-model mean from models driven by the SRES A1B.
  2. It appears that Chip did account for lag-1 autocorrelation when estimating his uncertainty intervals. So, his uncertainty intervals don’t suffer from that particular flaw. (Whether the simple method in Santer 2008 is adequate could be debated, but it does appear to have been applied.)
  3. It does look like both RC and Lindzen are doing some cherry picking of different sorts as suggested by Chip in his article.
  4. Trends computed based on observations are lower than the multi-model mean trend based on the A1B scenarios for all start years as far back as 1960. So, if we compare observations to the multi-model mean trend warming is “slower than expected”. No amount of cherry picking can make it “faster than expected.”
Written by lucia.

Comments

Nickle (Comment#21503)

Surely you should be regressing against log (co2 concentration), since the claim isn’t time causes warming?

Using log(c02), and current concentrations, with the 2C over the next 100 years, that gives a prediction in line with the claims.

Nick

Andrew_FL (Comment#21504)

Just think, we are 3 years from having so much non warming that even NOAA would have to say (if the used Hadley instead of their own data) that the models are off. Hadley shows at this point twelve full years without warming.

lucia (Comment#21505)

Andrew–
El Nino is here. Expect the number of years with zero trend to decrease. But then, we may bet a La Nina, and we shall see…..

George Tobin (Comment#21507)

Nick:
I thought the models had built-in assumptions about CO2 increase over time which is why we typically see temperature versus time presented. Why should lucia be required to plot temps against CO2 rather than time when none of the players in this instance (Chip, Gavin, Lindzen or Ramsdof) did so?

Is there any reason to believe that the CO2 levels predicted by the models are significantly wrong? Otherwise temp v. time seems a pretty straightforward and uncomplicated test of predictive power.

Nick (Comment#21508)

Lets say CO2 is linear, (it isn’t but its an approximation)

http://www.cgrer.uiowa.edu/new.....94/co2.gif

Shows an old graph.

http://earthobservatory.nasa.g.....aph_rt.gif

Is a more recent one.

Last 50 years linear is reasonable.

However the claim isn’t for a linear response to CO2. It is for an amplified log (co2) driver for the increase.

So using linear co2 is clearly wrong.

Nick

Zeke Hausfather (Comment#21509)

Lucia,

The +/- 90% range for the mean of models on the trend from 2005-present seems a tad small (looks like 0.15 to 0.35 eyeballing it) compared to the variability of individual model runs in that period, given that any individual model displays close to as much “weather” variability as actual temperature records.

See, for example, all non-volcano models from 2005 to present compared to GISS (the black line):


http://i81.photobucket.com/alb.....ure116.png

Perhaps this is a case where the mean of models doesn’t necessarily reflect weather noise better than any specific model run, since it cancels out?

lucia (Comment#21510)

Zeke–
This is the ±90% of in the estimate of the mean, as in a t-test. It’s not the model spread.

In a t-test, what you are testing is how much difference there would be in the estimate of the mean trend if you had a different collection of 22 models selected from the universe of all possible models.

Analogs in other things:
If you want to find the average height of men in the US and compare to the average height of men in Indian, you can find the average height for of American’s by measuring 500 men. You get the standarde devaition, sd. Then, you can compute the standard error in your estimate of the average by dividing the standard deviation by the square root of 500 men, se. Then, you can look up the multiplier for 90% , t. Then multiply t*se = the 90% confidence intervals.

This value can get arbitrarily small if you measure enough men. That’s the one I’m showing. So, it’s more or less what you are looking at divide by the square root of 22.

(One reason I show that is I’m usually interested in whether the mean trend predicted by models is biased high. But the other one is that if you put uncertainty intervals around the earth’s observations, you have to use this type of uncertainty, otherwise you are double counting.)

lucia (Comment#21511)

Nickle–
The climate models already account for the variation of CO2 with time parametrically. These aren’t steady state solution; they are time responses.

lucia (Comment#21512)

Zeke–

Perhaps this is a case where the mean of models doesn’t necessarily reflect weather noise better than any specific model run, since it cancels out?

Well… they don’t. But also, if the models did reflect the weather noise and weather noise was AR(1), the standard deviation over all models would match the standard deviation we estimate from a single observation of a time series.

But, in any case, the spread across models is due both to “model weather” and to biases in models giving different mean long term mean trends. I’ll find a quote from a paper I read just before going off to take care of my Dad. By the time we are discussing 10 year trends, most of the spread across model trends is due to differences in the mean response of models not weather noise. (I think that’s the cut-off. I’ll find it a bit later and show.)

Andrew_FL (Comment#21513)

Lucia-you make a good point. I’ll admit I can’t know if this period of little warming will end soon or stick around longer-If El Nino starts or is starting to take effect it probably won’t. But we will see!

Zeke Hausfather (Comment#21514)

Lucia,

Makes sense. However, prior to 2005 models track GISS and whatnot fairly well, and its mainly the low 2005-2009 temps that widens the mismatch between projections and observations. If ENSO or other factors that are present in individual model runs can replicate the sort of variability observed in the past 5 years, but that variability is averaged out in the model mean, couldn’t the variance of the estimate of the mean be a tad skewed in this sort of comparison?

My stats are unfortunately rusty, so let me know if that doesn’t make sense :-p

Zeke Hausfather (Comment#21515)

Actually, the temp/model mismatch was more noticeable prior to 2005 than I suspected by eyeballing the series. Interesting.

This shows GISS minus model mean (across all models):

http://i81.photobucket.com/alb.....ure117.png

lucia (Comment#21516)

Zeke–
You discovered part of my answer on your own. The model mean was a bit aggressive during the 80s and 90s too. It’s true that if the earth had “caught up” after 2005, we would not be talking about the models over predicting warming. But, it didn’t.

Nick (Comment#21517)

Lucia,

They are based on an amplified log(c02) model.

Clearly there is a time based component, but its going to be some sort of damped reponse to the forcing. It must be damped in order to be stable. Now, from the rapidity of falls in particular shows that the damping can’t last that long. ie. Look at the post 1998 fall. The fall can’t be caused by an increase in CO2. An increase could be caused by CO2 or some other factor. Hence there is information in the falls not contained in the rises.

Hence showing the model with CO2 as the explainatory variable is the better solution.

Nick

lucia (Comment#21518)

Nick–
You seem to be trying to explain internal variability as being caused by CO2. It’s not. The earth’s would have variable weather even if CO2 was constant.

The AOGCMs use CO2 to drive the temperature changes. That’s not the topic of this post.

Zeke Hausfather (Comment#21519)

Nick,

Given that ln(CO2) increases pretty monotonically, the results look nearly identical to those you obtain using years as the X axis. And as Lucia points out, the lag between a change in CO2 concentrations and equilibrium temperature make a direct comparison over shorter time periods fairly meaningless.

Nick (Comment#21520)

No, I’m not. There will be variability with the weather, not mater what.

However, the issue is that those claiming GW is caused by increases in CO2. They are not claiming GW is caused by the earth getting older.

Explainatory variable is CO2.

The physics that even the IPCC state is that heating is related to log (C02) concentration.

So unless you have some evidence that heating caused by CO2 is different than heating from such other source such as solar, I can’t see why you don’t use log(co2) instead of time as the explainatory variable

Nick

lucia (Comment#21521)

Nick–
Many questions can be asked. The graphs and analyses differ depending on which is being asked at any particular time. In this post, one question is:
“Models predict the rate of change in temperature in time is ‘blah’. The earth’s trend was observed to be ‘bleh’. Do these match?” For this question, you look at the rate of temperature as a function of time.

Of course one could also ask the sorts of questions you are asking, and for those, other graphs would make sense. But since those aren’t the topic of this post, those graphs do not appear here.

If you have a particular point you wish to make and the discussion would benefit from a particular graph, you are welcome to make your graph and tell us your idea.

Nick (Comment#21522)

I’ve no problem with that. CO2 data isn’t a problem, do you have a link for the temperature data?

Nick

lucia (Comment#21523)

Zeke Hausfather (Comment#21525)

Nick:

http://www.woodfortrees.org/da.....:-0.235875

gives you comparable temp data for all major series.

Steve Reynolds (Comment#21533)

I made a new 3 box model that seems to fit the most recent GISS data fairly well:
http://moderateclimate.blogspot.com/

lucia (Comment#21535)

SteveR–
Pretty cool! (I think I’m going to have to read up on Aikake (sp?) to learn now to compare whether more complicated models explain better than less complicated ones. That might be fun.)

hunter (Comment#21537)

As a rank lay person, the most important thing I see is that in any of the graphs, very little has actually happened.
Luke warm, indeed.

Steve Reynolds (Comment#21539)

Lucia,

Any thoughts on my speculations about the fit?
Has anyone made a model of ‘recovery from the LIA’, or do only denialists talk about that?

tetris (Comment#21540)

Warming, flat line or cooling. Even if, as Lucia posits, the warming continues but is slower than expected, why is that so? The “what” is always interesting, but the “why” is what it’s all about.

As others, like me, have suggested here before, it is the absence of a coherent answer from AGW/ACC proponents as to the “why” that is causing the entire topic to rapidly fall off the public/political radar screen.

lucia (Comment#21541)

SteveR–
I was going to look more tomorrow.

My impression is that all projections of the 20th century start with a “spin up” which by definition would eliminate any “recovery from the LIA” possibility from the results. I don’t know of anyone who has tried to set the ocean in some state that would later result in warming due to a recovery.

Mac (Comment#21550)

Trend Analysis is completely Bananas if you are dealing with purely chaotic climate events. The tendency of humans trying to create order out of disorder ultimately leads to ritual.

Just an observation!

DG (Comment#21556)

Despite recent spikes in LT (and surface for that matter) temperatures, the Levitus OHC update should dispel any notion the earth is “warming” or accumulating heat in the atmosphere. Once this heat loss from the oceans is transported from the atmosphere to space, the near surface air will respond in kind by plummeting. There is no other way to go. It may take another 6 months, but it will happen.

A similar event occurred last December with the sudden stratospheric warming (SSW) over the Arctic, but now it is a global phenomenon. What caused the oceans to have this step drop in heat content is open for debate, but it is what it is and the so-called unrealized heat “in the pipeline” theory never had a chance. The heat was always there, just not released yet into the air. It is now.

The SOI is back into positive range for Sept which indicates El Nino is struggling to survive, let alone strengthen. In fact, Australia is already reporting the beginnings of the formation of La Nina.
http://www.abc.net.au/news/sto.....395929.htm

This El Nino period is quite removed from previous others; it’s obvious. Sorry for going off topic, but I think it is important to look at the big picture of how the oceans mitigate global surface temperatures, not GHG. The OHC trend is now firmly in the negative, placing Hansen et al 2005 in file 13.

DB (Comment#21558)

“In fact, Australia is already reporting the beginnings of the formation of La Nina.”

Note that the link was to an article from October 2008.

John F. Pittman (Comment#21562)

Thanks SteveR. It looks like Fuller may win his Romm bet.

DeWitt Payne (Comment#21564)

As long as the subject of box models has been raised on this thread, I found this reference recently: North, et al, Simple Energy Balance Model Resolving the Seasons and the Continents: Application to the Astronomical Theory of the Ice Ages, JGU, 88, pp 6576-6586, 1983. It’s a two dimensional two box model with the heat capacity of the land being 1/2 an atmosphere column and 75 m water for the ocean. The continents have their current shape and position. Surface heat transfer is diffusive with the diffusion coefficient being a function of latitude. Abstract.

Chris (Comment#21582)

How come they didn’t go back 30 yrs (since the beginning of satellite data)? Great post, Lucia. Also, Mac nails it. Nick, asking fundamental science questions regarding climate models will only provide frustration. In addition to graphing temp anomalies versus log CO2 concentration, why not plot similar graphs for SO2 concentration, brown smog, ozone, atm opacity, cloud cover, ocean heat content, etc.? The answer is simple. To do so, the author would be protraying that the data is easily available and universally accepted. For sulfates and smoot (for example), no one has a clue as to what these values have been over last the 50 years. The fact that climate modelers are hiding their ignorance on such basic information behind these models is the real travesty. In other words, the modelers are obfuscating their lack of knowledge while at the same time portraying the opposite when publicizing the results. Shameful, and I hope in the next 10 years the public catches on to their shenanigans. Everyone at RC will get what they deserve.

Nick (Comment#21584)

Cloud cover is a consequence. Ocean heat content is a consequence. Therefore it wouldn’t make sense to use them as explainatory variables.

Things like Soot and sulphates have an effect. If we have solid data then they should be used, and excluded if they don’t produce a statistically significant effect.

Nick, asking fundamental science questions regarding climate models will only provide frustration.

To whom? Are you really trying to say there is no association of temperature against log (c02)? Are you trying to say that its some other relationship?

The log(co2) is well known and can be tested in the lab. Something rather unusual for climate science.

Nick

DeWitt Payne (Comment#21614)

Nick,

The IPCC says the scientific understanding of aerosols on climate is low. But you can’t get a high climate sensitivity and still hindcast the twentieth century temperatures unless you use a large aerosol effect to reduce the ghg forcing. Look at the size of the numbers in the last two columns (direct and indirect aerosol) in gissforc.txt. Graeme Stephens of Colorado State University, for one, says this about the indirect effect:

Observational inferences on indirect radiative forcing do not support the large values of forcings being applied in models. I would recommend model assessments be done with/without IRF [indirect radiative forcing]

Mac (Comment#21616)

The bottom line for me is that this planet’s climate is essentially random. There are chaotic events, and there seems to be periodic patterns in the data, but determining climatic trends is simply a time limited human response (a climate snap-shot).

If human beings could live for a 1,000 years we would be able to say with some certainty, “that’s the climate for you, one century freezing cold – the next boiling. You never know what you’ll get next.”

Chris (Comment#21634)

Nick,

If we could plot global temp versus 100 variables, we could develop a model based on data correlation alone, with some variables given more weighting than others. It could be based on neural nets, algorithms, or formulas (with or without fundamental properties). In engineering, most of the science was based on developing lots of data and lots of datasets. Emprical correlations came first, then followed by fundamentally-derived formulas, and lastly, predictions from (mostly) first-principal models. The climate modelers went straight to models after scarcely collecting any data. This is a recipe for disaster. And the people at RC, who I doubt ever designed a circuit, a bridge, an airplane, or a chemical process, can’t see the flaws in their models due to their outsized arrogance.

Recent Temperature Trends | Detached Ideas (Pingback#21655)

[...] The best discussion of this article is by Lucia here. [...]

Steve Reynolds (Comment#21696)

DeWitt Payne : “But you can’t get a high climate sensitivity and still hindcast the twentieth century temperatures unless you use a large aerosol effect to reduce the ghg forcing.”

Actually, I think you can. That is what I did with a 3 box model that retains a realitivly warm deep ocean from before the LIA. The forcings were directly from the GISS model.

A 1.8K climate sensitivity seems to fit pretty well:
http://moderateclimate.blogspo.....armer.html

Why Reconstructions Matter Part II « the Air Vent (Pingback#22504)

[...] skill in reproducing the observed mean climate. By http://masterresource.org/?p=5240at Lucia’s http://rankexploits.com/musing.....y-picking/ one can find an evaluation of the model’s recent skill. At [...]

Ron Cram (Comment#22658)

lucia,
El Nino may be here but we are now in a cool climate regime, the one predicted by the 2002 Bratcher and Giese paper would take place “in about four years.” So, they were off by a year. The shift actually took place at the end of 2007/beg of 2008. Notice how the satellite temps drop suddenly and arctic ice returns quickly. This is very similar to the climate regime shift that happened in 1945, right after the Northwest Passage had opened up in 1944. The shift to a warm climate regime in 1975/6, was just as dramatic as our more recent climate shift. Anyway, in a cool climate regime, El Nino will not be nearly as strong as it was during the warm climate regime. We should continue to have cool temps for another 30-35 years, as long as the pattern holds.

Al Tekhasski (Comment#22916)

Nick (Comment#21584) asks: “Are you really trying to say there is no association of temperature against log (c02)? Are you trying to say that its some other relationship? The log(co2) is well known and can be tested in the lab. “

Ok, let see what we have in widely accepted ice core record, which are considered as a benchmark for all climate studies. If you believe that Antarctic ice quantitatively preserves concentration of ancient CO2, and dO18 correctly represents global temperature, the Vostok ice cores show a correlation that can be interpolated as 8ppm/K in first approximation, as shown here:
http://www.ferdinand-engelbeen.....ation.html
These are the observational data, and they apparently include _all_ possible climate feedbacks, so no excuses here.

Now, if you assume (as IPCC and all AGW catastrophists) that CO2 is the main driver of temperatures, then the modern 375ppm of CO2 should force global temperature to rise for another 13.5K. Thus one would expect today’s global temperature to be 301.5K, which is not true by far. This simple estimate shows that the attribution of today’s warming to CO2 emissions is seriously flawed.

More, you say that log(CO2) is the correct coordinate, so increase in T must slow down with increase in CO2. However, if you examine the above chart, you would discover that the CO2-T relationship has a curvature that is _opposite_ of what one would expect from log(CO2). This is not just a value of some sensitivity coefficient that could be “corrected” and “re-analyzed”. This is a topological signature, which is impossible to reverse (unless you re-evaluate the entire basics behind ice core measurement technique.)

All this is really frustrating when the ends do not fit even for very basics in climatology.

 

Comments Closed: If you would like them re-opened, Contact Lucia