{"id":17961,"date":"2011-11-01T09:54:06","date_gmt":"2011-11-01T15:54:06","guid":{"rendered":"http:\/\/rankexploits.com\/musings\/?p=17961"},"modified":"2011-11-01T14:33:22","modified_gmt":"2011-11-01T20:33:22","slug":"response-to-keenan","status":"publish","type":"post","link":"https:\/\/rankexploits.com\/musings\/2011\/response-to-keenan\/","title":{"rendered":"Response to Keenan"},"content":{"rendered":"

Doug,
\nThanks for posting your email to me.<\/a> As I said, I will respond to whatever you are willing to post in public. I am taking the liberty to respond here where I can more easily use blockquotes.<\/p>\n

Lucia,<\/p>\n

Your recent blog post “BEST data: Trend looks statistically significant so far” includes the following statement.<\/p>\n

Since smoothing is discussed in Keenan’s letters to the economist, I will note that this data is computed by averaging over 12 months. So, it is “smoothed” relative to monthly data.<\/p><\/blockquote>\n

If we have a series of n monthly values, and obtain from that a series of n\/12 annual values, then we are not doing smoothing in the sense I intended; rather, we are doing aggregation. Aggregation is fine. Smoothing is problematic, and the problem is easy to understand.
\n <\/sub>
\nSuppose that our original series is a1<\/sub>, a2<\/sub>, a3<\/sub>, a4<\/sub>, …, an<\/sub>, and we are taking a 3-point moving average. Denote the smoothed series by b1<\/sub>, b2<\/sub>, b3<\/sub>, b4<\/sub>, …, b n-2<\/sub>. Then b1<\/sub> = (a1<\/sub> + a2<\/sub> + a3<\/sub>)\/3, and b2<\/sub> = (a2<\/sub> + a3<\/sub> + a4<\/sub>)\/3, etc. Notice that both b1<\/sub> and b2<\/sub> depend upon a2<\/sub> and a3<\/sub>. Hence b1<\/sub> and b2<\/sub> are correlated with each other. In other words, the smoothed series is more autocorrelated than the original series. That is the problem that smoothing (via a moving average) introduces. <\/p>\n

Suppose, on the other hand, that we simply aggregate the original series, to obtain c1<\/sub>, c2<\/sub>, c3<\/sub>, c4<\/sub>, …, cn<\/sub>\/12. Then c1<\/sub> = (a1<\/sub>+ a2<\/sub>+ a3<\/sub>+ … + a12<\/sub>)\/12 and c2<\/sub> = (a13<\/sub>+ a14<\/sub>+ a15<\/sub>+ … + a24<\/sub>)\/12, etc. Thus c1<\/sub> and c2<\/sub> do not depend upon the same elements of the original series. Ergo, the aggregation does not introduce auto-correlation.\n<\/p><\/blockquote>\n

I’m not entirely sure how whatever point you are making is supposed to relate to what I wrote in the context I wrote it. I was simply explaining what I did. I took unsmoothed data and stuffed it in your algorithm, computed the AICc criterion. <\/p>\n

Moreover, since, when criticizing Muller, you recently complained about smoothing, I didn’t smooth it. I stuffed the unsmoothed data into a process, computed AICc criterion and “Presto!”<\/p>\n

It seems to me you are now explaining that it was ok for you to smooth (by averaging) because you understand the problem introduced by smoothing and use it appropriately. In your email, you are describing this as a valid process:<\/p>\n

    \n
  1. smoothing over ‘n’ values<\/li>\n
  2. pulling out only every “nth<\/sup>“<\/li>\n
  3. give the full process of smoothing followed by culling the name of aggregation so as to distinguish it from smoothing and not culling.<\/li>\n<\/ol>\n

    Of course, many<\/em> types of aggregates exist: Aggregating functions include giving the count, finding a miximum or minimum, a sum, a mode and so on. Yours happens to be an aggregate of “n” sequential things– that is you created an aggregate obtained by averaging — which is smoothing. <\/p>\n

    You are also explaining that auto-correlation is introduced by performing a moving average: I believe many of us understood this long before you explained it. Maybe your point is to explain that when you<\/em>, Doug Keenan smooth, you<\/em> are careful to avoid pitfalls involved in smoothing so as to avoid fooling yourself.<\/p>\n

    If so, that is not<\/i> the message one would take home from the criticism you sent Richard Muller [BEST Scientific Director]; Charlotte Wickham [BEST Statistical Scientist]
    \nCc: James Astill; Elizabeth Muller. In that letter, you quoted Briggs and wrote:<\/p>\n

    Unless the data is measured with error, you never, ever, for no reason, under no threat, SMOOTH the series!<\/strong><\/p><\/blockquote>\n

    It seems to me that Muller made a point very similar to the one you are making now<\/em>, when in his response, he wrote<\/p>\n

    “He [Keenan] is, of course, being illogical. Just because smoothing can increase the probability of our fooling ourselves doesn’t mean that we did. There is real value to smoothing data, and yes, you have to beware of the traps, but if you are then there is a real advantage to doing that.”<\/p><\/blockquote>\n

    Maybe in future, when you post your criticisms of BEST, you can be a little more nuanced and explain the actual problem<\/em> that arose when BEST’s averaged and then did something not-permissible after averaging. That is: Explain how they used averaging and then failed to compensate<\/i>.<\/p>\n

    In your email to me you continue. <\/p>\n

    \n Your post also says this.<\/p>\n

    Doug’s claim that we might conclude the “IPCC assumption is insupportable” might be convincing to me if I bought for one second that it makes sense to use d=1 in ARIMA(3,1,0). I don’t. I think that arguments for statistical models with d=1 tend to violate the 1st law of thermodynamics.<\/p><\/blockquote>\n

    The physical plausibility of an ARIMA(p,1,q) could be questioned, at least on long time scales; on the other hand, ARIMA(p,1,q) might be a reasonable approximation, on time scales of interest here, to a more physically-plausible process. As an analogy, Earth is approximately spherical, but if someone is drawing a map of England, it is reasonable to assume flatness. Similarly, given the shortness of the time series, ARIMA(p,1,q) could be reasonable. <\/p><\/blockquote>\n

    First: I’m not saying “the physical plausibility of an ARIMA(p,1,q) could<\/em> be questioned”, I am<\/I> questioning it based on the laws of thermodynamics, heat transfer. (I have admitted to providing only a heuristic explanation. )<\/p>\n

    Second: Just because the surface of the earth can be approximated as flat even when it is spherical doesn’t mean something that is implausible at long time steps can magically become plausible at short time steps. You actually have to be able to give an explanation why an approximation might hold over a particular range<\/i>. <\/p>\n

    I question ARIMA(p,1,q) for natural forcings at all time scales. Moreover, to the extent that driftless<\/i> ARIMA(p,1,q) is put forward as contradicting that an apparent trend is due to anthropogenic forcings, I question d=1 even if you<\/i> have decided to provide a flimsy reason to to consider “driftless ARIMA(p,1,q) a candidate over the time span your consider doesn’t mean that anyone have to buy it. <\/p>\n

    I don’t buy your argument: I consider it flimsy. <\/p>\n

    Doug, you continue<\/p>\n

    In any case, the comparison of the ARIMA(p,1,q) model and the IPCC model strongly indicates that the IPCC model is failing to explain some substantial structural variation in the data–and that is the sole purpose of the comparison. Thus the comparison gives insight, regardless of physical plausibility.<\/p><\/blockquote>\n

    If, as you now claim, you were to limit your claim to saying that ARIMA(1,0,0) + trend is failing to explain the structure of the time series you would find few to contradict you. Many people were saying so at blogs, in journal articles and elsewhere long before you wrote your WSJ article. <\/p>\n

    However, if the “the sole purpose of the comparison” is to show the AR(1)+ trend model is “failing to explain some substantial structural variation in the data”, then the following claim in your WSJ article is rather overblown. <\/p>\n

    “… but the improved fit does tell us that until more research is done on the best assumptions to apply to global average temperature series, the IPCC’s conclusions about the significance of the temperature changes are unfounded. “<\/p><\/blockquote>\n

    The improved fit<\/I> shows research is needed? Lots of people are doing research and not because of your demonstration that some other ARIMA model fits the data better.<\/p>\n

    But beyond this, it is hardly the case that the IPCC’s conclusions about the significance of temperature changes rests solely, or even principally<\/em> on the AR(1)+ trend model. Much of the IPCC’s confidence in the significance is based on non-statistical arguments– including comparisons of outcomes between models driven by GHG’s and those not. It’s true some readers here might not accept the IPCC conclusions and don’t favor modeling approaches. But showing that the AR(1)+trend model is fails to explain structural variation barely puts a dent in their argument that the trend is attributable to anthropogenic forcings because that’s not their main argument<\/i>. <\/p>\n

    Moreover, that you (or many others) can find and present a<\/I> model a driftless ARIMA model with a better AICc coefficient than ‘AR(1)+trend’ doesn’t overturn<\/em> the IPCC’s conclusions of the significance of temperature changes. We can easily find other ARIMA models with either drift or trend (e.g. ARIMA(0,1,4)+drift) with better AICc’s than the one you highlighted in the supporting materials for your WSJ. If the argument is to pick out based on AICc, (and I see little other in your<\/em> WSJ article) the IPCC’s conclusions seems to remain founded. The only criticism would be that the discussion of the AR(1)+ trend model isn’t particularly useful but we would still<\/em> conclude there was either drift or trend if we look for the ARIMA(p,d,q) model with the best AICc criteria. <\/p>\n

    You, Doug, go on:<\/p>\n

    Your post further says the following.
    \nThat is: using the preliminary BEST data going back to 1800, the IPCC AR(1) blows the model favored by Doug Keenan in his Wall Street Journal out of the water. The model that says “Statistically significant warming” wins.<\/p>\n

    Update(May 25): If I use annual averaged best data, the model that wins reverses. As already promised below, I’ll be discussing other models.<\/p>\n

    In the first case, you were using seasonal (monthly) data, which cannot be directly compared like that–as your Update effectively showed.\n<\/p><\/blockquote>\n

    Correct. I was using monthly data — as would be required if it were actually true<\/i> that one should “never, ever, for no reason, under no threat, SMOOTH the series!” <\/p>\n

    Naturally, since the admonition to never<\/strong><\/em> smooth is invalid, I acknowledged the difficulty when people requested I used the smoothed — i.e. averaged data. In fact, I expect people would ask me to do so because it’s actually ok to smooth!<\/i>. <\/p>\n

    You, Doug go on:<\/p>\n

    Additionally, the ARIMA model was not “favored” by me; it was solely used for comparison with the IPCC model. Indeed, the WSJ piece ended by saying that more, and difficult, research was required.<\/p><\/blockquote>\n

    First: With regard to your current claim that the ARIMA model was “solely used for comparison with the IPCC model”, it appears that your criticism in the letter to the Economist makes a much more expansive claims for your article in the WSJ where you write<\/p>\n

    To summarize, most research on global warming relies on a statistical model that should not be used. This invalidates much of the analysis done on global warming. I published an op-ed piece in the Wall Street Journal<\/a> to explain these issues, in plain English, this year.<\/p><\/blockquote>\n

    In this, you certainly appear<\/em> to be claiming that your op-ed piece in the Wall Street Journal communicates the fact that much of the analysis done on global warming is invalid! <\/p>\n

    Either– as you now claim– the only thing your article shows is that the AR(1)+trend model “is failing to explain some substantial structural variation in the data”, or your op-ed piece does something that seems to indicate that “much of the analysis on global warming [is invalid]”. (I should note that, in my opinion, the wording of your WSJ claim gives the reader the impression that the better AICc for the driftless<\/I> ARIMA model does more than merely point out a structural inadequacy in the AR(1)+trend model. )<\/p>\n

    Second: I an mystified by your objection my using the verb “favor” to describe your characterization of the ARIMA(3,1,0) model over the ARIMA (1,0,0)+trend model. <\/p>\n

    You<\/i> elected to use the ARIMA(3,1,0) model rather than some other as the example in the supporting materials for your WSJ article. You<\/I> bring up AICc as a criteria to pick on model over another. You<\/i> guide the reader to the notion that the ARIMA(3,1,0) is better by this metric. That fits the definition of “favoring” it.<\/p>\n

    At least as far as supplying the reader with information, you use ARIMA(3,1,0) model as “the” model to support this statement in your WSJ article<\/p>\n

    “A fairly elementary alternative assumption that some researchers and I have tested fits the actual temperature data better than the IPCC’s AR1 assumption?so much better that we can conclude that the IPCC’s assumption has no support. Under the<\/strong> alternative assumption, the data do not show a significant increase in global temperatures.”<\/p><\/blockquote>\n

    So, even though alternatives exist that have a better fit than driftless ARIMA(3,1,0), you happened to pick this<\/em> one- to highlight in your WSJ article and supporting materials. And you use the definite article “the” not “this” or “an”– which suggests to naive readers that there might only be two alternatives. But even if you’d used “this” or “an”, you focused on the existence of a particular alternative<\/I> one and advise it<\/em> is better than AR(1)+trend. You don’t mention the possiblity of ARIMA(0,1,4) with drift<\/em> which means there is a secular warming trend– and you don’t mention this despite the fact that it’s whose AICc beats ARIMA(3,1,0). <\/p>\n

    I would say the choices you made when writing your WSJ article and defending ARIMA(p,d=1,q) constitutes your favoring<\/em> ARIMA(3,1,0) both over AR(1)+trend but even over other alternatives with d=1. <\/p>\n

    But beyond that in your email to me (posted here) you defended the ARIMA(p,1,q) model against criticism that it is physically implausible thus giving the impression that you do, indeed, “favor” — that is show a preference– for this model above a number of other models. <\/p>\n

    I want to add this: I realize in your WSJ you write this,<\/p>\n

    We don’t know whether the alternative assumption itself is reasonable?other assumptions might be even better?but the improved fit does tell us that until more research is done on the best assumptions to apply to global average temperature series, the IPCC’s conclusions about the significance of the temperature changes are unfounded.<\/p><\/blockquote>\n

    Yet, for some reason, you object to my saying that we do<\/I> know whether “the” alternative referred to in your WSJ article and in your supporting materials is reasonable. What we know is that physical arguments would lead us to say that it implausible if we are going to attribute warming to natural forcings<\/i>. <\/p>\n

    That is: if d=1, then it must be because the anthropogenic forcings are d=1, because “natural forcings” cannot be. Of course, if d=1, because<\/i> anthropogenic forcings are driving temperature changes, this tends to support the notion that current temperature are high because<\/I> ghg’s have increased. In this case your the closing sentence of your article “the IPCC’s conclusions about the significance of the temperature changes are unfounded.” is false. Because finding d=1– if that was “the” only alternative to AR(1)+ trend would indicate that the rise cannot result from natural forcings<\/i>. <\/p>\n

    It may well be true that my argument isn’t based on statistics<\/i> but on physics, but there is absolutely no reason why those trained in the physical sciences and who are familiar with thermodynamics cannot say that they have “issues” with someone presenting an analysis that highlights a statistical model that appears to violate physics. <\/p>\n

    To summarize, your criticisms are invalid.<\/p>\n

    Sincerely, Doug<\/p><\/blockquote>\n

    I think it’s pretty silly to decree one’s own arguments victorious. I happen to think most of what you write is pretty weak. But that’s my opinion and I will leave it to others to decide what they think of your criticism of best, your claims that you don’t “favor” the model you highlighted, and your defense of putting forward ARIMA(p,1,q) models. I’ve said I don’t like d=1, I think averaging can be used– carefully– and I you certainly give the impression that you favor–i.e. prefer– ARIMA(3,1,0) to AR(1)+trend.<\/p>\n

    Update<\/b>: 3:30 pm. Doug Keenan emailed requesting I edit to create subscripts in the first quote. I did so.<\/p>\n","protected":false},"excerpt":{"rendered":"

    Doug, Thanks for posting your email to me. As I said, I will respond to whatever you are willing to post in public. I am taking the liberty to respond here where I can more easily use blockquotes. Lucia, Your recent blog post “BEST data: Trend looks statistically significant so far” includes the following statement. … Continue reading Response to Keenan<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15],"tags":[],"class_list":["post-17961","post","type-post","status-publish","format-standard","hentry","category-data-comparisons"],"_links":{"self":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/posts\/17961","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/comments?post=17961"}],"version-history":[{"count":0,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/posts\/17961\/revisions"}],"wp:attachment":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/media?parent=17961"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/categories?post=17961"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/tags?post=17961"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}