AJ noticed I’d used the Climate Explorer incorrection [blush]. This is the ENSO correction using SST’s in the actual honest to goodness cold tongue:
In this: The Knight et al. update has trends back to 1984 inside their HadCM3 uncertainty intervals. I have not yet obtained the model data for the 10 runs they used so these are crude “Method of the Eyeball” uncertainties picked off their graph.
Meanwhile I am trying to:
- Obtain data for those 10 runs. (Heck… identify them!) I’ve asked John Kennedy the 2nd author. He’s just back from Taiwan, so I doubt this is is highest priority. (His highest priority may be getting over jet lag and/or saying hi to the wife and kids.) Richard Betts has forwarded my request to Jeff Knight. But anyone who thinks they know where I can get the 10 runs– feel free to help out– and please be as specific as possible.
- Verify my hunch that Thompson first detrends cold tongue SST’s before computing ENSO temperatures. (He later detrends ENSO before computing a correlation with HadCrut4. So, in my currently case, I think order of operations mostly only matters if the ENSO temps are computed over a longer period than used to find the correlation between ENSO temperatures and HadCrut4, but I’d prefer to know.)
Today, I’m going to see if I can down the SST values for the AR4 models and compute the enso temperatures for the models. As a short cut, I’m going to use the cold tongue heat capacity for the earth with the models. Later on, I’ll try to do the best fit for each model, and fix that. The goal of this exercise is that if ENSO correction is good for Knight et al, it ought to be “good” for the AR4 models. Whether the method is valid or not, it will to reduce the size of the uncertainty intervals in my previous analyses which would result in tests with the appearance of greater statistical power. (If the method is not valid, the appearance of greater power will be deceptive. It will merely be overfitting. But… it should be possible to do some tests for that.)
KnightEtAll_ENSO_Adjust
Time now goes backwards? HadCrut4 has dropped half a degree C in 30 years?
Edit: I think I don’t quite understand what you’re doing here, I’m finding the presentation confusing.
redc, you’re reading the graph backwards. It shows the trend in anomaly since the given year.
E.G. The uncorrected (red) trend since 1992 is approx. 0.3C, i.e. 0.15C/decade.
I worked it out but found some of the labelling confusing. Maybe time and/or trend on the y-axis would’ve helped me.
I am having a conceptual problem here. How does shifting temperature anomolies around in time make any sense? Aren’t they saying, after the fact, that the reason our projections are wrong is because the actual observations are wrong?
Doesn’t this problem instead illustrate one of the deficiencies with the current global anomalies system? A huge gaping hole in the system?
I tend to think that both the models and observations are flawed.
redc –
The vertical axis is labeled “Temperature Change (C)” in Knight et al. [Change the “(C)” to “(°C)” or “(K)” if you’re feeling pedantic.]
Genghis–
Do you mean my shifting the temperature changes to the left? Yes. That makes sense. Do you mean something else? If so, clarify. Then I can explain.
On the other questions:
No, no and no.
Harold–I’ll try to remember to change that in the script. But… I think I found a problem with their method.
Lucia,
Like I said I am having a little conceptual difficulty here. : )
If I understand it correctly (doubtful) they are saying that if it wasn’t for the ENSO ( a temperature anomaly sloshing around in the ocean) the global temperature anomaly would have risen in accordance with the model projections?
So what they are trying to do is calculate the actual global anomaly with the ENSO anamoly properly accounted for, lowering the past global anomaly and raising the current global anomaly? Thus demonstrating that there has been no pause in Global warming.
Heads up See Tom Nelson CG3 has been initiated by FOIA
http://tomnelson.blogspot.com/2013/03/mr-foia-speaks-time-to-tie-up-loose.html
Lucia… FWIW, I was able to run your code down to the point where the ENSO correction was calculated (I didn’t attempt to go any further). As you probably did, I visually compared the correction to Knight’s figure and it looked like the peaks and valleys were in the right place.
I was interested in comparing Knight’s correction to my own made-up method (using NINO3.4 and arbitrary parameters). They were highly correlated with a coefficient of 0.88 and with Knight’s lagging mine by 2 months.
I then did a ccf of both of these on the HadCRUT4 quadratically detrended anomalies. The correlation of Knight’s method maxed out at 0.57, lagging HadCRUT4 by 1 month. My method maxed out at 0.52 with a zero month lag.
Given that Knight’s coefficient beats mine by a fair margin and the lags are comparable, I’ll say that it doesn’t suck.
AJ –
The algorithm for ENSO adjustment is not credited to Knight et al., but to Thompson et al. It may or may not be significant that Kennedy was co-author on both articles.
Thanks Harold… I’ll rephrase: “… the Thompson correction doesn’t suck as much as mine” 🙂
Who Hoooo!!!!!!
HaroldW–
Yes. But it is the *application* of the correction to *problem* addressed in Knight et al that causes a problem. And… there is problem *as applied in Knight* when b≠0.
Here are print outs from a “toy” problem… I’m going to be coy– because I can’t explain in a comment. I need figures and such. But:
> beta;
[1] 0
> sqrt(mean_var)*10; # stdev over decade.
[1] 0.1135088
>
> sqrt(mean_var_corrected)*10; # stdev over decade.
[1] 0.05546595
Note: For beta =0, the sqrt(mean_var_corrected)< sqrt(mean_var)*10;
Which was probably Knight and authors goal when enso correcting. They probably wanted to *reduce* the variability in trends (i.e. mean_var) across their ensemble by taking ENSO out. If you could find a method that *really did that* without introducing “features”, then… well.. that would make this method *great*. And it’s why when we started discussing Knight et al after Lord Higt Liberal Arts Major Monckton showed up, I thought (to myself) hmm… I should not only check Knight with it’s own data, but I should see what it does on AR4 data. Because maybe I can apply it to AR4 models data and reduce my error bars on some of the graphs I show comparing observations to model spread.
Well. I tried the Knight method on GISS_ER data. (Because I could get 9 runs from pre-industrial.) I got
sqrt(mean_var_corrected) > sqrt(mean_var)
Huh? Why? Bug hunting. Typo hunting. Cursing at my spaghetti code. Adding in figures. Then… I plotted something. And AHA!!!
I got a germ of an idea: The method — as applied in Knight to get uncertainty intervals– gets messed up as a result of non-linearities in the expected value of the hindcast. And I kept thinking of it after going to bed. (Jim is in California….. Kept muttering…. And the germ of idea grew into an infection of an idea.)
So… I decided to code up a “toy” problem. The toy problem has nonlinearities in the *expected value* of the surface temperature and cold tongue SSTS– as measured. From a statistics point of view, these non-linearities are “dirt”. To introduce the “dirt” I made the hindcast in the “toy” model parabolic. Here is a result for a ridiculous amount of curvature– large beta:
> beta;
[1] 10
> sqrt(mean_var)*10; # stdev over decade.
[1] 0.1105393
>
> sqrt(mean_var_corrected)*10; # stdev over decade.
[1] 0.1770642
Note: for beta≠0, sqrt(mean_var_corrected)*10 >sqrt(mean_var)*10;
There are actually three parameters in the toy model. It’s a toy so I can provide a qualitative demonstration of ‘the problem’.
But these were my first two not-crashing runs and I wanted to see a “clear” result quickly without running too much monte-carlo. While typing the comment I did beta=5 > beta;
[1] 5
> sqrt(mean_var)*10; # stdev over decade.
[1] 0.1059720
>
> sqrt(mean_var_corrected)*10; # stdev over decade.
[1] 0.1292836
(You probably think: Lucia must be done coding? But no…. I need to show that not only do your estimate of the width of the uncertainty bars increase when computed from an ensemble, the variability computed over 1 single time trace decreases so the effect on your stated p values takes a double whammy!!!! Yowsa! )
But… I’m going to give it a rest. The cats are bugging me. I’m going to have pizza and watch tv.
Congratutations Lucia! I have no idea what you’re talking about. Looking forward to the pictures.
AJ–
Does NINO3.4 use surface temperatures when computed? If yes, it might have the same problem. You can’t diagnose the problem by looking at how “well” it works on a single time series.
The only ‘correction’ methods that would not have the feature I described are those parameterize ENSO based something other than temperature. For example: wind speed in Darwin or some such.
The problem doesn’t arise because Thompson’s method is bad at diagnosing ENSO. But like all methods, it’s imperfect. And they way in which it’s imperfect can cause problems if it’s used as in Knight et al.
When I write this up, I’m going to discuss:
1) Why it would seem like a good idea when you first think it up. In fact, the beta=0 results make it seem like a truly brilliant idea. If one had a problem with beta=0, this method would be a champ.
2) What happens if beta≠0 in my toy problems. Why one might not think of this. Why– if you don’t think of the possibility of the problem, you might not look for it. Why/when the problem “happens”.
4) Hmm… the only real question I have… is the one I asked Thompson. But I’m pretty sure… that .. the problem exists unless he defined “SST” anomalies in a non-characteristic way and then didn’t say. He’s out of town until tomorrow!! (Also, John Kennedy may be able to tell me. He was just back from Taiwan, but maybe he’ll be able to answer soon.)
5) Just how bad it can be in “toy” problems.
6) Provide evidence that it *does* happen in relevant to Knight et al cases– using GISS_ER which I did last night.
Lucia,
“I’m going to be coy”
Ok. I feel better about not following AT ALL.
Hi Lucia,
Sounds intriguing. I was guessing that the problem was going to be something like the relationship between the ENSO-like features of the model and its average temperature, doesn’t behave as in the real world. So the Thompson approach to ENSO adjustment doesn’t actually cause the models’ adjusted avg surface temperature to have the same variability as the observed average, resulting in incorrect confidence intervals. [Or, duration of low/insignificant slope necessary to demonstrate that the models are inconsistent with observations, for those who like to play that game.]
SteveF–
It’s going to take a whole bunch of blog posts!! I’m trying to decide on the best order.
The best order might be to show *what happens* with GISSER first. And then explain why!
HaroldW– The reason is very “weird”! And yet… obvious once it’s understood.
AJ, SteveF-
Me either (not following) but in my case that’s not too surprising. My statistics are rustier than my calculus, and that’s saying something. Usually takes me quite a bit of muttering and head scratching and googling stuff I’m pretty sure I was supposed to know for a test 20 years ago and so on to follow Lucia even when she draws helpful diagrams and spells stuff out. 🙂 Still, I’m not complaining. I learn more here than I do on any other blog I read, provided I spend the time and effort to try to follow along.
Looking forward to reading about it Lucia! It’s nice to escape the cg3 speculation madness for a bit.
Mark… I feel your pain, only my tests were closer to 30 years ago. I feel a certain reward though when I re-learn and re-understand something long forgotten.