Friday, I promised you all several boring posts, and now I’m going to deliver!

Today’s post will contain equations relating the variance of an averaged process to the underlying instantaneous process with exponentially decaying autocorrelation. The discussion will ignore the existence of the mean trend. (This simplification involves no loss of generality when later use the functions to relate statistical properties of monthly averages to annual averages.)

Because I haven’t installed proper math functionality at the blog, will not include any derivations, only results. But, if I don’t write up results, I won’t remember then 2 weeks from now when someone posts a comment asking me what equations I used in a spread sheet!

Continuous AR(1) process

Let us imagine that weather noise v(t), is a continuous stationary process of t, with mean zero (i.e. 〈v(t)〉=0), variance σ²= 〈v(t)v(t)〉 and autocorrelation described by the function ρ=〈v(t)v(t+s)〉 = e^|-s/Ï„|.

(Note: 〈〉 brackets are used to indicate “ensemble average” of any quantity inside the brackets. So, 〈Y〉 is the “ensemble average of Y”.)

The parameter Ï„ is a time constant, describing a period of time during which the correlation between noise at time “t” and that at time “t+s” drops by a factor of e^-1. In the case of AR(1) noise, the quantity Ï„ corresponds to something called the “integral time scale” for the weather-noise. A large time constant corresponds to weather that is, on average, quite persistent. So, for example, if Ï„ were very large — say 10 years– and the earth’s surface was unusually warm today, one would expect the surface to remain unusually warm for 10 years. In contrast, a a short time constant would correspond to weather varying rapidly. So, if Ï„ were equal to 1 hours, today’s relative warm might dissipate in an hour or so.

Variances of process averaged over time T.

Suppose an agency measures ‘v’ and reports monthly, or annual averages? How do the variances of these averaged measurements relate to the variance of the underlying process, σ²?

Well… if I’d properly installed math functions, I’d show you the derivation. But, instead, I’ll just define the meaning of symbols and show the result.

Let’s call the variance of the “v” averaged over “T”, σ_T². If we average “v” in windows with block “T”, and variance of this quantity is related to σ² as follows:

(1) σ_T²= σ² [ 2 Ï„/T ] [ 1 + Ï„/T {exp(-T/Ï„) -1} ]

It is possible to show that this equation has the following correct properties:
1) σ_T²= σ² as T/Ï„ → 0.
2) σ_T²~ C /T, where C is a constant as Ï„/T → 0, as required for white noise.

Sneaking in just a little of the debate over testing the AR4 2C/century projection

Since the purpose of this exercise is to see test whether the AR(1) process Gavin considers “closer” to the models than the one I obtain by fitting data, I’ll now do a bit of fiddling to convert the process Gavin described into a monthly process. (Mind you, I’m under the impression gavin doesn’t believe the models are AR(1), but I still think it’s interesting to test whether the AR(1) process that he thinks closer to the models falls within the range of possible AR(1) processes for the data. )

So, let’s turn to the process Gavin described as being closer to representative of models, which Gavin described as follows

However, for the case that is closer to what I did with the models, I calculated 9000 7-year AR(1) time-series with an underlying trend=2 degC/century and with p=0.1.

After I asked him, Gavin said the he used a standard deviation of σ_YEAR=0.1C for this year.

To test whether Gavin’s AR(1) process reproduces “weather noise”, or trends consistent with data, I’m going to test the hypothesis that his entire process describes weather against data. However, to do the test, I must translate his process from annual to monthly.

If Gavin’s “closer” process were not averaged, and ρ described the autocorrelation for the continuous process at a 1 month lag, then to determine the time constant of the process prior to averaging, we would use:

(2)ρ = e^|-s/τ|.

Evaluating for s=1 year and ρ =0.1 we get Ï„ = 0.44 years. (You can double check: 0.1 = exp(-1 year/ 0.44 years. 🙂 )

For this case, the variance of the annual averaged quantity is found using equation 1, and we obtain σ_YEAR²=0.53 σ², which is to say, by averaging over a year, the variance of the “weather noise” drops by about 1/2 compared to the continuous process.

In contrast, if we average over a month, the variance of the “weather noise” will be σ_month²=0.92σ², which is quite a bit larger.

So, it turns out that ifτ = 0.44 years, the ratio of the standard deviation of the monthly averaged measured value to the annual standard deviation is: σ_month/σ_YEAR= 1.3.

So, based on what you’ve seen so far, you might imagine that to translates Gavin’s process to monthly values, I should use σ_month= 0.13C.

But this won’t be quite right because, as I noted before, I found the time constant Ï„ by ignoring the fact that Gavin’s ρ_YEAR corresponds to an annual averaged process. This means I can’t use equation (2) to obtain a precise value of Ï„ for the continuous process. I need to do a bit more fiddling!

How far has this boring post gotten us?

So, far, I have an equation that lets me find σ_month that corresponds to σ_year if I know Ï„. I have an approximate way to estimate Ï„ from the annual averaged values– but it happens that I know it’s not quite right.

So, in the next boring post in this series, I’ll describe how to find the Ï„ that matches the annual averaged process. I’ll might also discuss a few other issues related to using avearaged data instead of instantaneous data (including why being forced to use averaged data can be a pain in the ***)

After that: on to monte-carlo and hypothesis tests! 🙂