{"id":22043,"date":"2013-04-12T10:58:33","date_gmt":"2013-04-12T16:58:33","guid":{"rendered":"http:\/\/rankexploits.com\/musings\/?p=22043"},"modified":"2013-04-13T12:27:22","modified_gmt":"2013-04-13T18:27:22","slug":"the-truth-is-out-there-comment-on-the-dimple-part-i-of-iii","status":"publish","type":"post","link":"https:\/\/rankexploits.com\/musings\/2013\/the-truth-is-out-there-comment-on-the-dimple-part-i-of-iii\/","title":{"rendered":"“The truth is out there” : Comment on the Dimple Part I of III"},"content":{"rendered":"

It is often good to focus on “The Truth” which in this post we will represent using $latex T_{true} $. We will also call this truth “the measurand”. With “The Truth” in mind, we will discuss the error in a measurement, apply that definition to define an error in a proxy reconstruction and and see if we can learn something Marcott’s Dimple<\/a>.<\/p>\n

\"uncertainty\"<\/a><\/p>\n

In the figure above, we see a plot that shows “Uncertainty” in some unstated thing. Blog arguments have broken out; I think it’s worth considering how we would fill in the blank in “Uncertainty in _____”. I’ll begin with this question:<\/p>\n

What is the measurement error<\/em> in a proxy reconstruction for the surface temperature of the earth?<\/b>
\nLet us consider the goal of such a reconstruction which is to estimate or measure the surface temperature of the earth, $latex T_{true}(t) $ which varies over time t. The proxy reconstruction is an estimate which we will represent as $latex T_{rec}(t) $. Given this definition of what we wish to measure, the error in the proxy reconstruction, $latex e_{rec} $, can be determined by applying the definition of a measurement error. That is we subtract our estimate from the thing we wish to measure. <\/p>\n

(1)<\/eqnumber>$latex \\displaystyle e_{rec}(t) = T_{rec}(t)- T_{true}(t) $ <\/equation><\/p>\n

The confidence interval for the measurement error $latex e_{rec}(t) $, is defined as the standard deviation across proxies of $latex e_{rec}(t) $ which we will represent as $latex \\sigma_{e}(t) $. I claim confidence intervals based on this $latex \\sigma_{e}(t) $ is the proper definition of “the measurement uncertainty” for the proxy reconstruction. <\/p>\n

Before proceeding, it’s worth considering two example properties of confidence intervals described by $latex \\sigma_{e}(t) $. <\/p>\n

Feature 1: provided the temperature of ‘the earth’ $latex T_{true} $ is defined using a consistent baseline that is not affected choice of proxy, then the measurand, $latex T_{true}(t) $ is effectively deterministic. In this case, as long as $latex T_{rec}(t) $ from all possible proxies share a common baseline, $latex \\sigma_{e}(t) = \\sigma_{T_{rec}}(t) $ should be identical.<\/p>\n

Feature 2: Since the motivation for the Marcott proxy reconstruction is to discover changes<\/I> in temperatures of the earths surface over time, it is worth considering how the uncertainty intervals in (1) can be used to estimate the uncertainty in $latex \\Delta T_{true} =[ T_{true}(t_{2}) – T_{true}(t_{1})] $ for an arbitrary choice of $latex t_{1} $ and $latex t_{2} $ . <\/p>\n

Notice that if $latex T_{true} $ and $latex T_{rec} $ are both white noise, using definition of error in (1) has desirable property that the error for $latex \\Delta T_{true} =[ T_{true}(t_{2}) – T_{true}(t_{1})] $ is $latex \\sqrt{ [ e_{rec}^{2}(t_{2}) – e_{rec}^{2}(t_{1})] }$ Consequently, uncertainty and confidence intervals defined based on the standard deviation of these errors will exhibit this property one anticipates for uncertainty and confidence intervals of a measurement<\/i> of $latex T_{true} $. (They will also share other properties one anticipates for confidence intervals of measurement<\/I> uncertainties.) This means these confidence intervals are useful<\/I> if we wish to make claims like that the one the one that follows:<\/p>\n

” Global temperatures are warmer than at any time in at least 4,000 years, scientists reported Thursday, and over the coming decades are likely to surpass levels not seen on the planet since before the last ice age. ” <\/a><\/p><\/blockquote>\n

See:New York Tmes<\/a><\/p>\n

They are also useful if one merely wishes to know the uncertainty in when one compares the earth temperature in 1850 to the temperature at other times during the Holocene. <\/p>\n

Other confidence intervals<\/b>
\nBecause our motive is to discuss alternate views of possible definitions of “confidence intervals for measurement<\/em> uncertainty in a proxy reconstruction”, I will discuss a second set of confidence intervals discussed at some length by Nick Stokes<\/a>. He seems to call these “confidence intervals [for some unstated property]<\/font> of the Holocene temperature reconstruction”. (Note, I inserted the words in blue). His confidence intervals (for this unstated property) are defined as the standard deviation on $latex T_{rec, rebaselined}(t) $ which is $latex T_{rec}(t) $ from an individual reconstruction that has been rebaselined using the the mean of $latex T_{rec}(t) $ computed using reconstructed temperatures from that individual reconstruction over the baseline chosen for a particular analysis; we will denote the standard deviation for this quantity as $latex \\sigma_{Trec,rb}(t) $.<\/p>\n

For now I would like to highlight this feature of the confidence intervals for (some unstated thing): The measurand, $latex T_{true}(t) $ for the proxy reconstruction does not appear the definition. Of course, if we compute a standard deviation on $latex T_{rec, rebaselined}(t) $ we will obtain confidence intervals for $latex T_{rec, rebaselined}(t) $ which is something<\/I>. So Nick is describing confidence intervals<\/i> in the measurement of something<\/i>. <\/p>\n

However, it is rather difficult to use these intervals to compute the uncertainty in $latex \\Delta T_{true} $ above. (We will later see that if you try to use them for that purpose, you do so incorrectly.)<\/p>\n

The Toy Problem<\/b>
\nI will now set up and discuss a toy problem which will show that — at least in the simplest problem the confidence intervals for $latex \\sigma_{Trec,rb}(t) $ show a “dimple” in the baseline period, while confidence intervals for $latex \\sigma_{e}(t) $ describing measurement uncertainty<\/i> have no dimple. I’m not going to go so far as to claim that confidence intervals describing measurement errors can never<\/i> have a dimple. It may be that under certain conditions they do. However, in this simple case, the dimple does not appear in confidence intervals for measurements errors<\/i>. <\/p>\n

True temperatures in ‘The Toy’<\/b>
\nTo set up the toy I will posit that the true earth temperature is measured at evenly spaced intervals $latex t_{i}$ and the measurements $latex T_{true}(t_{i}) $ are Gaussian white noise with mean 0 and standard deviation 1. The seeming contradiction that from the point of view of a proxy reconstruction the earth temperature is deterministic will be resolved by decreeing that the temperature already exists and once generated is considered “frozen”. The goal of proxy reconstruction is to figure out what those temperatures are. <\/p>\n

We will decompose the temperature $latex T_{true}(t_{i}) $ into the mean computed during a baseline defined “N” specifically selected points in time, $latex \\overline{T_{base}} $, and a residual $latex \\overline{T(t_{i})} $ . The decomposition is: <\/p>\n

(2)<\/eqnumber>$latex \\displaystyle T_{true}(t_{i}) = T(t_{i}) + \\overline{T_{base}} $ <\/equation><\/p>\n

Henceforth the overline $latex \\overline{X} $ will be used to denote the sample average of X over whatever period is defined as the baseline for a proxy reconstruction. The subscript “base” above will communicate the same notion; its use above is redundant. (The redundancy may be helpful to those who wish to read the accompanying code.)<\/p>\n

Note that outside the baseline $latex T(t_{i}) $ is a random variable with mean $latex -\\overline{T_{base}} $ and standard deviation 1, while inside the baseline it is a random variable with a mean 0. <\/p>\n

Proxies in ‘The Toy’<\/b>
\nNext suppose we can obtain an estimate of $latex T_{true}(t_{i}) $ from available proxies. There are many and any individual proxy will be denoted with a subscript ‘j’. The raw proxy value of each proxy will be assumed to vary linear with true temperature as<\/p>\n

(3)<\/eqnumber>$latex \\displaystyle P_{j}(t_{i}) = m [T(t_{i}) + \\overline{T_{base}}] + W_{j}(t_{i}) + P_{mean,j} $ <\/equation><\/p>\n

where $latex m $ is the linear conversion constant which to simplyfy the analysis will be considered common to all proxies, and $latex W_{j} $ is Gaussian white noise with variance $latex B $ and zero mean. $latex P_{mean,j} $ as a constant which may be specific to the individual proxy. It is introduced to account for the effect of calibration biases of unknown magnitude in proxy ‘j’. For perfectly calibrated proxies, the term could be set to zero would have no effect on the value of $latex \\sigma_{e}(t) $. <\/p>\n

Let us now define<\/i> the raw reconstructed<\/del> proxy calibrated temperature based on a proxy ‘j’ to be
\n(4)<\/eqnumber>$latex \\displaystyle T_{rec,j}(t_{i}) = P_{j}(t_{i})\/m $ <\/equation><\/p>\n

Error in the reconstruction and Uncertainty Intervals in ‘The Toy’.<\/b>
\nWith a small amount of algebra it is possible to show that the difference
\n(5)<\/eqnumber>
\n$latex \\displaystyle e_j{rec}(t) = T_{rec,j}(t_{i}) -[T(t_{i}) + \\overline{T_{base}}] = W_{j}(t_{i}) + P_{mean,j}\/m $ <\/p>\n

<\/equation><\/p>\n

If we like baselines, we could define $latex T_{rec,j}(t_{i}) $ over the baseline as $latex T_{rec,j} $ and subtract write (5) above as <\/p>\n

(6)<\/eqnumber>$latex \\displaystyle e_{rec,j}(t_{i}) = [ T_{rec,j}(t_{i}) -\\overline{T_{rec,j}} ] -[T(t_{i}) -\\overline{T_{rec,j}}] ] – \\overline{T_{base}} = W_{j}(t_{i}) + [P_{mean,j}\/m] $ <\/equation><\/p>\n

Recognizing that both $latex \\overline{T_{base}} $ and $latex T(t_{i}) $ are unaffected by choice of proxy, j, the standard deviation of $latex e_{rec,j}(t_{i}) $ (i.e. $latex \\sigma_{e}(t) $ ) over all possible proxies is unaffected by the magnitude of either. Their effect will be to shift every<\/I> point in a reconstruction up or down uniformly by some unknown temperature. This is called a bias error<\/I> and the standard deviation in that quantity would represent an uncertainty in the bias. <\/p>\n

Diagnosing the contribution of $latex [P_{mean,j}\/m] $ to $latex \\sigma_{e}(t) $ is a bit more difficult. If all proxies responses are similar to a perfectly calibrated laboratory thermometers and the true surface temperature of the earth was known, then we would anticipate that $latex [P_{mean,j}\/m] $ would be effectively deterministic for a proxy reconstruction. (Specifically, the analyst creating the reconstruction could subtract out value at the “true” baseline temperature for the earth and it the term could be forced to zero for every proxy.) So the the term would make no contribution to $latex \\sigma_{e}(t) $. <\/p>\n

However, if a thermometer is more like single “treenometer” or<\/I> the true surface temperature of the earth can never be known, then $latex [P_{mean,j}\/m] $ may include sizable random component that corresponds to different calibration errors both for true thermometers and “treenometers”. The magnitude of that calibration error will depend both on the noisiness of the ‘treenometer’ itself (i.e. the magnitude of “B” for the proxy), the number of data points used to calibrate the individual ‘treenometer and the uncertainty in the true temperature of the earth. <\/p>\n

However, the effect of $latex P_{mean,j}\/m $ is irrelevant to the question of “The Dimple”. So in the (not yet written part II) of this discussion, I will show graphs of the uncertainty computed with $latex P_{mean,j}\/m $ set to a deterministic value. This decision means that I will be finding the <\/p>\n

  • lower bound<\/i> on the estimate of the variance in the measurement errors in the reconstruction. In (as yet not written) part III I will compute bounds varying its magnitude.\n

    Before moving on, let’s consider some features of confidence intervals based on (5) or (6) . In ‘The Toy’ problem neither $latex P_{mean,j}\/m $ nor the standard deviation of $latex W_{j}(t_{i}) $ are functions of time, therefor the $latex \\sigma_{e}(t) $ is not a function of time. So $latex \\sigma_{e}(t) $ will contain no dimple in this problem.<\/i> <\/p>\n

    Nick’s uncertainty intervals in the toy<\/b>.
    \nFinally, it’s useful to examine the properties of Nick’s confidence intervals for some unstated thing<\/font>. Recall his confidence intervals are defined as the standard deviation of rebaselined reconstructions. Using the overbar terminology that means his he is taking the standard deviation of:<\/p>\n

    \nIn the original, I have an error. I’m revising the bits inside the grey boxes.<\/b>
    \n(7original)<\/eqnumber> $latex T_{rec,rb}(t_{i}) = [ T_{rec,j}(t_{i})-\\overline{T_{rec,j}}] = W_{j}(t_{i}) – \\overline{ W_{j} } $ <\/equation><\/p>\n

    If we take the standard deviation of (7) to obtain $latex \\sigma_{Trec,rb}(t_{i}) $ we will find that both terms on the right hand side contribute to this standard deviation. In this case, $latex \\sigma_{Trec,rb}(t_{i}) $ is<\/i> a function of time. For points inside the baseline, $latex W_{j}(t_{i})$ and $latex \\overline{ W_{j} } $ are positively correlated and $latex \\sigma_{Trec,rb}(t_{i}) $ will be smaller than the variance of $latex W_{j}(t_{i})$. For points outside the baseline, the two noise terms are uncorrelated; the computed variance will be larger than $latex W_{j}(t_{i})$.\n<\/div>\n

    ——–<\/p>\n

    (7 corrected)<\/eqnumber> $latex T_{rec,rb}(t_{i}) = [ T_{rec,j}(t_{i})-\\overline{T_{rec,j}}] = T(t_{i}) + W_{j}(t_{i}) – \\overline{ W_{j} } $ <\/equation><\/p>\n

    From this we can identify a differen measurement error which is
    \n(8)<\/eqnumber> $latex e_{rec,anom}= T_{rec,rb}(t_{i})- T(t_{i}) = W_{j}(t_{i}) – \\overline{ W_{j} } $ <\/equation><\/p>\n

    If we take the standard deviation of (8) to obtain $latex \\sigma_{Trec,rb}(t_{i}) $ we will find that the two terms on the right hand side contribute to this standard deviation. In this case, $latex \\sigma_{Trec,rb}(t_{i}) $ is<\/i> a function of time. For points inside the baseline, $latex W_{j}(t_{i})$ and $latex \\overline{ W_{j} } $ are positively correlated and $latex \\sigma_{Trec,rb}(t_{i}) $ will be smaller than the variance of $latex W_{j}(t_{i})$. For points outside the baseline, the two noise terms are uncorrelated; the computed variance will be larger than $latex W_{j}(t_{i})$.<\/p>\n

    This is the origin of the dimple which does appear in the sorts of confidence intervals Nick is computing but does not<\/i> appear in confidence intervals describing measurement errors of absolute temperatures<\/i>. <\/p>\n

    Nick will be happy to know that I now see a “measurement error” because it is expressed relative to a measurand!<\/font> (Whether this measurement error is properly interpreted when these are compared to temperatures in the thermometer record or projections I cannot say because we now have two types of measurement errors but which is most easily used when patching to a thermometer record I don’t yet know.)<\/p>\n

    Before continuing on to the exciting plots (to appear in Part II), I think it’s also worth noting that $latex P_{mean,j}\/m $ does not appear in (7 or 8). <\/p>\n

    \nThe following is largely right, but interpretation has to change. Missing that term does affect comparisons to a temperature (or anomaly) at a time outside time period of the reconstruction but possibly not two terms inside the reconstruction<\/i> <\/p>\n

    This means that whatever (7) is supposed to account for, it does not<\/i> capture the effect of calibration bias in proxies on the uncertainty in measurement errors in proxy reconstructions. For this reason alone, whatever the confidence intervals might be what they are not<\/I> is confidence intervals that can be used to compute the uncertainty in $latex \\Delta T_{true} =[ T_{true}(t_{2}) – T_{true}(t_{1})] $ .<\/div>\n

    I don’t know if confidence intervals computed the way Nick is computing confidence intervals were applied to estimate the uncertainty in $latex \\Delta T_{true} $ in Marcott. But if the claims about temperature differences relative to those in the thermometer record<\/font> made in Marcott are based on that sort<\/I> of confidence intervals, those claims would be based on misinterpreting what confidence intervals computed the “Nick” way describe. Remedying this mistake would require computing confidence intervals for the error in the reconstruction<\/I>. A proper estimate would require capturing the effect of the calibration uncertainties described by $latex P_{mean,j}\/m $. <\/p>\n

    Summary: Part I<\/b>
    \nSince I’m going to be showing graphs in separate posts it’s useful to summarize the main points in this post:<\/p>\n

      \n
    1. The uncertainty intervals computed using (1) correspond to the uncertainty in the error<\/i> in a reconstruction. These are useful if you wish to determine the uncertainty in the difference in the temperatures at two different times in the earth’s history. Learning this difference seems to be the goal of a proxy reconstruction, so these confidence intervals are useful<\/i> and describe the uncertainty in the measurand of interest<\/i>. These uncertainty intervals will not show “The Dimple”. I call these proper “measurement uncertainties” for a proxy reconstruction because they can be used to estimate the uncertainty in the measurand of interest (i.e. the changes in the temperature of the earth.) <\/li>\n
    2. The uncertainty intervals computed using the method discussed by Nick in his blog post are not<\/em><\/del> may not<\/font> be useful if we wish to estimate the uncertainty in the difference in the temperatures at two different times in the earth’s history when one of those times lies outside the range of the proxy reconstruction<\/font>. If we base claims about the statistical significance of differences in temperature on that sort of confidence interval, your conclusions will have no legitimate foundation and — unless we are sufficiently luck– our claims will be incorrect. These confidence intervals will show “The Dimple”.<\/li>\n
    3. Computation of the two sets of confidence intervals share some features. In certain limits, the quantitive difference between the two will be imperceptible and conclusions based on the “Nick” type intervals be nearly identical to those made with proper uncertainty intervals. This happens only when both the following are true: The proxy reconstruction used a baseline of sufficiently long duration to make “The Dimple” so small as to be imperceptible and the proxies calibration is perfect. However, the fact that “The Dimple” appears is sufficient evidence to demonstrate this goal has not been achieved. <\/li>\n
    4. I have no idea whether Marcott’s claims about the uncertainty in temperature changes relative to the thermometer record<\/font> were based on uncertainty intervals computed as in the top figure in this post. I know those uncertainty intervals are described, but not having read the paper, I don’t know if those uncertainty intervals were used without modification<\/i> to estimate the uncertainty in difference in the earth’s temperature at different points in time nor do I know whether those uncertainty intervals were places around the mean reconstructions in their figure of the reconstruction. (If they were, that choice would be misleading.) Because I don’t know whether they did these things, I cannot say whether how Marcott might have used<\/I> confidence intervals computed the “Nick” way, I can’t say whether what they did was “right” or “wrong”. I can say nevertheless say that if<\/i> they used that sort of confidence interval to compute the uncertainty interval for temperature difference at different times on earth or if<\/I> the used slapped those sorts of confidence intervals around the mean reconstruction and represented that as the uncertainty in the measurement of the earth temperature, that would be misleading to the point of being incorrect. <\/li>\n<\/ol>\n

      Upcoming: Graphs showing “The Dimple” as it appears in “Nick-type” confidence intervals if the temperature series and the proxy noise are Gaussian white noise. Afterwards, graphs showing the relative size of confidence intervals using selected values chosen to highlight qualitative effects of interest which may not correspond to values relevant to Marcott. Those of you who like math and have been reading Marcott can suggest reasonable values for the random componenent of $latex P_{mean,j}\/m $, and the noise in the proxy reconstruction and so forth. <\/p>\n

      Teaser graph<\/b>
      \n      \"RejectionRates\"<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

      It is often good to focus on “The Truth” which in this post we will represent using $latex T_{true} $. We will also call this truth “the measurand”. With “The Truth” in mind, we will discuss the error in a measurement, apply that definition to define an error in a proxy reconstruction and and see … Continue reading “The truth is out there” : Comment on the Dimple Part I of III<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15,314],"tags":[273],"class_list":["post-22043","post","type-post","status-publish","format-standard","hentry","category-data-comparisons","category-toy-physics","tag-reconstructions"],"_links":{"self":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/posts\/22043","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/comments?post=22043"}],"version-history":[{"count":0,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/posts\/22043\/revisions"}],"wp:attachment":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/media?parent=22043"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/categories?post=22043"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/tags?post=22043"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}