Two days now and my comment still hasn’t cleared moderation.

# Charlie Says: Your comment is awaiting moderation.
30 June 2009 at 8:09 PM

I notice that several comments submitted have been approved since I sent in the one below. I’m assuming that it has dropped through the cracks somehow and am submitting it again:

Charlie Says: Your comment is awaiting moderation.
28 June 2009 at 7:01 PM
Inline comment to #363: “Not sure why a small technical error in the caption would give ammunition to anyone except conspiracy theorists: ”

If you look at the URLs below, it is obvious that changing the smoothing period and end conditions radically changes the trend line of Figure 3 in the Copenhagen report.

See http://landshape.org/enm/anoth.....#more-2459 and
http://rankexploits.com/musing.....is-report/

The Copenhagen report was intended to show the affect of more recent data and studies. In keeping with that philosophy it would perhaps been better to leave the smoothing at the same 11 year setting that was chosen for the original report.

The only apparent advantage of changing the data smoothing is that it causes the trendline to continue without the more recent data having any significant effect on the plot. See the above URLs to see the changes.

There are a zillion possible ways to smooth, and it appears that the climatologists use many. The reason for any particular choice is obscure. I’m perfectly willing to believe any one of them might be useful in some specific situation. But with Rahmstorf, the choice of smoothing with any end condition to guess data makes the method specifically unsuited for testing fidelity projections.

Once again, we replicated Mannian smoothly exactly at CA in our analysis of Mann et al 2008 where we had before and after results – so there’s no replication mystery on this.

Mannian smoothing considers three different situations: 1) reflection; 2) reflection plus flipping; 3) projection of mean and then picks one of the three methods. This are given inflated names like Minimum Roughness etc.

All three “cases” are built into Mannian smoothing.

As I recall, reflect-and-flip corresponds to Emanuel’s bin-and-pin smoothing in his hurricane data, which he later renounced.

The method used in Rahmstorf 2007 is to append a fitted line through the m point onto the end of the series. Clearly the fitted line could be produced in a number of ways, n-order polynomial or SSA itself. In this case a first order polynomial, or linear fit is used. The different approaches illustrate the ‘model uncertainty’ around the choice of end treatments.

It think Mannian filtfilt is another method entirely, as Rahmstorf 2007 uses SSA. Its worth mentioning that the trend line is based on the first EOF of the SSA decomposition. So there is another question, how many EOF’s should have been used. Because SSA assumes stationary, periodic components, it tends to fit a linear trend with TWO periodic components (see my recent posts). So attempting to use the first component of SSA as a smooth is bound to arbitrarily introduce a second derivative, a curving up or down.

Anyway you think of it, its a dogs breakfast, when a simple modified t test would have been adequate.

Steve, I think filtfilt is a different smoothing method from SSA. Another Mannian smoothing method.

It’s as good as any, but has the fault that the most recent value is replicated m/2 times in the data – ie it has an artificially amplified effect.

This is not what MRC does. What MRC involves flipping the data with two mirrors.

The first flip is to assume the future data is the reflection of with mirror about the vertical. So, a set of three data with “y” values of y1, y2,y3 ending in y3 would be assumed to be y1,y2,y3,y2, y1.

Then after that the guessed data flipped around the “x” axis. So, if we had “rebaselined” such that y3=0, the data would be y1, y2, 0, -y2, -y1.

This tends to maintain whatever the trend existed was over the past “m/2″ data points and is not anymore sensitive to specific final point than any other point in the series.

The reason the 11 -year smoothing with mrc roughness goes flat toward the end is that (for whatever reason) the trend based on observations from 2004-2008 is not positive.

If climate modelers believed a claim that ‘weather noise” permits 20 year negative trends to be embedded within rapidly rising long term trends, then they can’t avoid flat spots appearing due to “weather noise” unless they use filters that smooth with time periods greater than 40 years. Of course, in that case, they can’t “test” any model projections (or even most hindcasts) from any if the IPCC reports by eyeballing graphs with smoothed data because everything from 1979 forward is in the “end region” and subject to change when new data comes in.

(Mind you, the could still test using other, not-foolish methods, like t-tests. But Rahmstorf’s choice of smoothing, which seems to involve selecting “m” after looking at the data, is nuts.)

I created an animated GIF for people to see how Stefan’s method of choice is behaving:
http://i39.tinypic.com/6fnvqa.gif

For those who have not followed this closely, M=11 (2006) “nonlinear trends” are the ones used in Rahmstorf et al Science-paper (2007), where they write:

“The global mean surface temperature increase (land and ocean combined) in both the NASA GISS data set and the Hadley Centre/
Climatic Research Unit data set is 0.33°C for the 16 years since 1990, which is in the upper part of the range projected by the IPCC. Given the relatively short 16-year time period considered, it will be difficult to establish the reasons for this relatively rapid warming, although there are only a few likely possibilities. The first candidate reason is intrinsic variability within the climate system. A second candidate is climate forcings other than CO2: Although the concentration of other greenhouse gases has risen more slowly than assumed in the IPCC scenarios, an aerosol cooling smaller than expected is a possible cause of the extra warming. A third candidate is an underestimation of the climate sensitivity to CO2 (i.e., model error).”

Of course, the fourth candidate might have been simply the “trend” calculation method they used…

Now M=15 (2008) corresponds to Figure 3 of the (Copenhagen) Synthesis report. There it has been written
“Figure 3 … shows the long-term trend of increasing temperature is clear and the trajectory of atmospheric temperature at the Earth’s surface is proceeding within the range of IPCC projections. ”

Of course, it was a simple mistake that the figure caption says
“All trends are nonlinear trend lines and are computed with an embedding period of _11 years_ and a minimum roughness criterion at the end “

Andrew

my comments were an expression of potential bias accepted by the community than a specific accusation

Yes. IANL. Still, expressing the opinion that a community could develop a bias could not remotely be called libel under US libel laws as I understand them.

Maybe in other countries? Who knows. But my impression (based on spelling and writing style) is you are American, Deep Climate is probably American, and we are all hosted in the US. So, what gives?

I rather thought my comments were an expression of potential bias accepted by the community than a specific accusation. His article accused certain types of folks known as ‘deniers’ as being biased. My point is that it was a two way street.

At the time, I wasn’t as pissed as I am today. The climate bill is a nightmare of bias and exaggeration and it’s almost enough to turn me into a denier monster.

The only thing holding me back is some warming makes sense. In 10 months of reading and study, I’ve seen no papers which show negative feedback has been dis-proven or positive is proven. The modeling shows too much warming compared to measured data (Thanks for your work on this). The surface temperature data is corrupt. Beyond that nobody really knows.

I wouldn’t have minded finding out AGW was true, at least we would know something. I don’t know if you’re aware, but I own a ‘green’ energy saving company and have already saved more CO2 for earth than all the greenies I’ve ever met. So there is some potential I could monetarily benefit from these idiotic laws.

Oh well…

Yeah. I read that blog post. People can abuse polynomial fits. Duh.He comes up with his own examples and suggest claims someone might have made, but no one did. Then he explains why the fictional people who might have made these claims would have been wrong had they made the wrong claim. M’kay.

He had a post demonstrating that someone had manipulated an polynomial curve fit look like more of a downtrend. I left a reasonable comment (for me ) which agreed with him but pointed out that his own extrapolation of a high order polynomial curve beyond the data was not a good way to demonstrate his point. Polynomials are after all, unconstrained after the end of the data. Here are some of the statements captain Deep (his moniker is way to cocky) had a problem with.

1. “Of course I have seen the binomial smoothing including some of the highly manipulable versions which straighten endpoints based on user input parameters. …These useful tools are also abused by people who want to make the data look different than it is.”

2. “It’s equally obvious that other blogs/scientists would insist that you can only use a linear fit when it shows an upward trend obfuscating a downward curve which is slight but real.

Well it’s pretty clear that is what’s been done in this post here but there are many examples we’ve all seen of the same behavior. After one day, I don’t comment at captain deep’s site anymore.

In the case of the Synthesis Report, the method was changed deliberately to keep an upward trend going. The change was then misrepresented in at least two places in the report.

With the exception of TCO-like entities, I think it’s pointless to do much snippage or heavy moderation. People would still say what they think somewhere else.

On the more substantive issue: Climate science does seem to resort to all sorts of arbitrarily selected filtering methods with a wide variety of endpoint treatments. I’m still wondering why JeanS’s comment didn’t appear at RC. Looked pretty tame to me.

I missed this one somehow before. Dr. Deep doesn’t seem to like any of the math you do Lucia, perhaps you should quit. The reason I bring him up is because he snipped one of my comments for pointing out the biased treatment of endpoint filtering in climate science. Here we see another glaring example. (please don’t snip me )

I wonder how they get away with it?

Its a wonder why they would use this method when perfectly good statistical methods exist for answering the questions of trend agreement, as your careful and rigorous analyses have demonstrated to everyone. Its a wonder why I see comments on the blogs that Rahmstorf has done it the ‘right’ way. Its a wonder anyone takes it seriously.