UPDATE:
To help visualize the difference between the satellite reconstruction – which is wholly mathematical – I spliced the manned station data and AWS reconstruction data into the grid (after centering). These two plots show:
1. The dichotomy between the trends from the actual data/AWS reconstruction and the satellite reconstruction.
2. The actual data/AWS reconstruction shows <i>cooling</i> in the main continent and <i>warming</i> in the peninsula only since 1969, whereas the satellite reconstruction shows warming everywhere (but very little warming in the peninsula).
I would argue that there is very little agreement between the actual data/AWS reconstruction and the satellite reconstruction.
UPDATE TWO:Â RUN YOUR OWN R MOVIE (no, not that kind . . . )
http://www.mediafire.com/?imkymyqz4fw
http://www.mediafire.com/file/mnyw15yy51m/spliceall.txt
http://www.mediafire.com/file/5qdzcuxynut/splicemanned.txt
http://www.mediafire.com/file/emtwyjmnzzd/Tir_lons.txt
http://www.mediafire.com/file/z0jy31xzjhb/Tir_lats.txt
To use the script above, you need the mapproj and maps packages from CRAN. Save Tir_lons and Tir_lats into your working directory. Load spliceall.txt and splicemanned.txt into R. Type:
sat_recon=spliced.manned (or spliced.all)
Then copy and paste the script and you can watch how the satellite recon evolves compared to the AWS and manned stations. To change how many months of averaging you’re doing, change the value of x. Additional instructions are in the last 4 lines. 🙂
ORIGINAL POST:
I thought I’d open Part Three with a couple of plots I made while trying to determine if any actual station data was included in the reconstructions. This is because some of the discussion gets technical, and I decided to use pretty pictures to try to sucker you in. 🙂
The plots above show the difference in trend between the satellite reconstruction and the actual surface station data for common points during the listed periods. There is a distinct difference. When compared to the surface station data, the satellite reconstruction shows less warming over much of the continent prior to 1980 and shows more warming after 1980 – with the notable exception of the Peninsula. This makes us ask two questions:
1. Why?
2. Is the difference statistically significant?
Before we can answer those, we must make sure we understand how the reconstruction was done.
MATHEMAGIC
The initial impression I had based on the article was that the satellite reconstruction used manned station data exclusively prior to 1982 and satellite information after 1982. The open question (I thought) was whether the manned station data continued post 1982. Determining this was fairly simple: I found the grid coordinates in the satellite reconstruction that corresponded to the surface station locations and differenced them. If actual station data was present, the result would have been a line along zero.
The result was definitely not a line along zero:
There was no station data present post-1982. In fact, there was no station data pre-1982, either.
While I was mystified, other, more intelligent folk (Roman M, Jeff C, and Jeff Id) over at CA and the Air Vent had already figured out what had been done. Here’s the CliffsNotes version:
1. Processed satellite data was combined with manned surface station data.
2. RegEM was used to infill missing values.
3. From that result, 3 principal components were calculated.
4. Using the 3 principal components, the entire continent was infilled.
For those who don’t know, principal component analysis is a method to break down a data set with many dimensions into fewer dimensions. In this case, the grid has 5509 cells, each with 600 points. Many of the time series are correlated to each other. The idea of principal component analysis is to break the large matrix of correlated data into a smaller matrix of the uncorrelated components – the principal components from which the rest of the data can be mathematically derived.
(While I understand the concept, I do not claim to understand the math. I will not be able to answer any questions you have about the theory supporting this type of analysis, except to say that it is a well-established and accepted form of analysis.)
What this means in English is that the satellite reconstruction is entirely mathematical. There is no actual data present anywhere in the reconstruction. It was derived entirely from the three principal components calculated in the previous step. If proper care is taken, this method will yield valid results.
Now that we know how the reconstruction was done, we can answer our two questions.
Question 1:Â Why are the trends different?
To put it bluntly, the answer to this question is because the data is not shared between the time series of the actual station temperatures and the time series of the corresponding grid point.
This does not necessarily signal a problem. Time series of monthly temperature anomalies are quite noisy. Performing linear trends on subsets of the time series can yield different results even when the series are well-matched. In order to know if we have a problem, we have to determine if the difference in trends are statistically significant. This leads us to . . .
Question 2:Â Is the difference statistically significant?
According to the paper, no. However, a closer look at the verification statistics used in the paper shows them to be insufficient to make this determination if the purpose is to trend subsets (specific time periods) of the data.
The three statistics Steig uses are r^2 (coefficient of determination), RE (reduction of error), and CE (coefficient of efficiency). RE and CE are calculated as follows:
where x-sub-i is the actual data, x-hat is what the reconstruction predicted for that point,
x-sub-C is the mean of the data in the calibration period, and
x-sub-V is the mean of the data in the verification period
RE and CE scores of 1.0 indicate perfect fits. As the fit worsens, RE and CE trend downward. Steig uses these statistics to ensure that the reconstructed temperatures are less than 1% likely to have arisen by chance from a red noise process. In other words:
Null hypothesis:Â Reconstructed temperatures are not distinguishable from a red noise process at a significance level of 1%.
Steig then runs 1,000 Monte Carlo simulations with red noise at each gridpoint to determine the RE and CE values that would arise from such a process. The cutoff point for a valid reconstruction is set at the 99th percentile of the RE and CE values obtained for the Monte Carlo simulations. If a given reconstructed time series achieves higher RE and CE scores than this cutoff, then the null hypothesis is rejected and the time series is verified. This shows that the correlations used to obtain the reconstructed values are not spurious – and this part is key – in the aggregate.
While this provides good assurance that the overall trend is not simply the result of a random process, it provides no insight into how the actual data (x-sub-i) and the reconstructed data (x-hat) evolve with respect to time. In other words, it does not provide verification that the reconstructed data yields trends that are statistically equivalent to the actual data.
The above plot are the smoothed trends for Amundsen Scott, which passed Steig’s verification. I used the loess function in R with a span of 0.5 to generate the smooth. The endpoints are distorted a bit, but the middle 40 years is represented reasonably well for the purposes of this example. Note how, if you subset the data, you can obtain very different trends between the two graphs. Because Steig’s verification statistics do not evaluate the fit between the data sets as they evolve with time, they cannot detect statistically significant changes in subsets of the data.
To detect significantly significant differences in subsets, a different statistical tool is required. For this purpose, I chose to evaluate how the means of the data evolve with time. It is computationally intensive, but it is a much more rigorous test.
The test I perform is a paired Wilcoxon test at each point in time using a 96-month span (48 months on either side). This is a generalized analogue to the t-test, but does not require the assumption that the distribution of the differences is normal.  So for a 600-month time series like the Steig reconstruction, I perform 600 Wilcoxon tests for each station (60,000 total tests to run it on all stations used). I then normalize the result to the 95% confidence interval and plot.
What this means is that any time the difference in means exceeds the 95% confidence interval, the resulting value will be plotted outside the range of +/-1. If the difference in means is not statistically significant, it will fall inside that range. The reason for normalizing this way is to be able to plot multiple series on the same set of axes.
The results are not good for Steig.
The first set is a selection of comparisons to actual manned station data. The second set is a selection of the same comparison to the AWS reconstruction.
Some of the series spend more time outside the confidence interval than inside.
WHAT DOES THIS MEAN?
With this more sensitive test, we find that every single series with greater than 30 years of data spends a statistically unlikely amount of time outside the confidence intervals. This allows us to reject our null hypothesis that there is no statistically significant difference in the evolution of the data sets with time.
How does this affect Steig’s results?
The most noticeable aspect of this is the behavior of the peninsula stations. Remember the plots at the beginning of this post? They showed blue dots on the peninsula – indicating that the satellite reconstruction was understating the warming. Looking at the peninsula plots above, we see that in early times, the satellite reconstruction is too warm and in late times it is too cold compared to the actual data.
The second aspect is that the satellite reconstruction overstates the warming for the remainder of the continent. Note how the vast majority of the series are outside the lower 95% confidence level in the 1980-1990 period and many exceed the upper 95% confidence level after the year 2000. This results in a trend that is too high.
In fact, the situation is even worse for Steig than it first appears. Remember that our test looks at the previous 4 years as well as the subsequent 4 years. A linear trend starting in 1980 is unaffected by previous data. If we were to change our test to be forward-looking only (to be consistent with a linear trend), we would find that our low peak shifts 4 years earlier – and the linear trend starts right smack in the middle of it. In some cases, this results in the change in temperature over the trend period being nearly a degree C too high.
In other words, the null hypothesis that there is no statistically significant difference between the satellite reconstruction and the ground data/AWS reconstruction is strongly rejected.
I will leave this here for the moment. There is a third question that is more fundamental than the two we just answered, but it is late and this post is already way too long. 🙂
When was the satellite data changed so it more closely matched surface data?
I am impressed by the type of analysis you are attempting here. Seems like something the authors could have done to give more confidence in their methods (even if the “math” is right, one must have an objectively verified method — which seems to lack in climatology). Anyhow, this is one of the “value added” blogs because its authors make efforts like this one — thus, I visit every day.
Well, I hope the sequel is as good as the original.
There’s a typo in “principle component analysis”: it, and other uses of “principle” should be changed to “principal.”
Re: Roger Knights: Oops! All fixed. 😉
.
Re: Mark R: Without the processed AVHRR data, it is not possible to tell exactly how or over what period the satellite data was calibrated to the ground data. It is also not possible to tell how much (if any) adjustments were made.
Isn’t the summary of this futile exercise that the surface data is spatially too sparse to obtain meaningful results for the entire continent? (Note that the satellite temperature retrieval algorithm is an ill posed problem because one is attempting to invert an integral.)
Jerry
From Watts up with….
“UPDATE: An additional question has been brought up related to why the data seems to be missing from the poles. Dr. Christy also responded:
As the spacecraft rolls over the pole it does so at an inclined orbit so
that the highest nadir latitude is about 82 deg with the scanner looking
out a bit closer to the pole. Since we apply the scan line data mostly to
the nadir area directly below the satellite, the actual data only go to
about 83 deg. In the gridded data I interpolate over the pole, but I
wouldn’t trust the data too much beyond 85 deg.”
http://wattsupwiththat.com/2008/03/08/putting-a-myth-about-uah-and-rss-satellite-data-to-rest/
Is there any actual satellite data at the poles?
Gerald Browning (Comment#10582)
Isn’t the summary of this futile exercise that the surface data is spatially too sparse to obtain meaningful results for the entire continent?
I’m inclined to guess that you’re right – or at least that we should be cautious as to how robust any such assessment might be.
It’s interesting to me that so much energy has been expended examining the issue following the publication of a paper finding a warming trend, when the previous meme of ‘Antarctica cooling’ received no such challenge. ‘Scepticism’ appears to me to be inclined in one direction!
Simon,
Given how pathetically bad Mann’s other work has turned out, it shouldn’t be a surprise that people check it out. Are there similar red flags about the cooling? That interpretation was based on a simple look at the data. And both alarmists and skeptics were in agreement about the cooling. Given all the areas of disagreement ripe for review or audit, why would anyone spend time on an area of agreement?
The “warming” conclusion, however, was dependent upon Mannian statistics to “discover”.
stan (Comment#10588)
Given how pathetically bad Mann’s other work has turned out, it shouldn’t be a surprise that people check it out.
So that’s an expression of confirmation bias, yes?
I’m not going to rerun the whole h-s debate here but, just for the record, I do not agree with your premise above. Never mind.
Are there similar red flags about the cooling? That interpretation was based on a simple look at the data
Whoa a minute – Antarctic ‘cooling’ (East Antarctica really) was extrapolated from sparse data points. That was not ‘simple’, or if it was then Steig’s extrapolation is simple.
And both alarmists and skeptics were in agreement about the cooling.
Not so simple in fact, but never mind. You are saying now that “skeptics” are not in agreement about warming? Is that a policy decision? I think you reveal exactly what you mean when you coin the term “skeptic”. I will point out to you that I do not use the term “denier” and I consider that your pejorative use of the term “alarmist” to describe all of those who take a different view from you is evidence of the fact that you are engaging in propaganda and not discussing science objectively.
You are spouting the familiar character-dissing approach to the discussion of these matters. If you want to say that something is wrong, misleading, or whatever, by demonstration of the content of what has been said, then do so. Dismissing work on the back of prejudices you’ve picked up elsewhere is another matter.
Frankly you have some chutzpah terming yourself a ‘skeptic’ when you’ve written such an explicit defence of confirmation bias.
Simon Evans:
No, its an expectation of a continuous trend from the HS to the present day. Mann’s statistical cluelessness and poor data analysis has been a constant over time, so its no wonder that every time Mann pops up with yet another statistical model wrapped in an enigma, people are expecting errors.
As for whether Mann’s HS work was rubbish, then I would have to refer to the analysis done by Wegman’s team as well as known commentaries by Jolliffe, an expert on PCA. Wegman called it simply “bad science” and the mathematical treatment simply “wrong”. Jolliffe called it “a piece of dubious statistics”.
You know better? Present the evidence.
Otherwise I think you’re simply projecting your own prejudices upon someone else.
John A (Comment#10591)
its an expectation of a continuous trend from the HS to the present day. Mann’s statistical cluelessness and poor data analysis has been a constant over time, so its no wonder that every time Mann pops up with yet another statistical model wrapped in an enigma, people are expecting errors.
Precisely as I said, then – an expression of confirmation bias (expecting a trend to continue and interpreting further evidence in the context of such an expectation is, precisely, confirmation bias).
Thanks for the confirmation 😉
As for whether Mann’s HS work was rubbish, then I would have to refer to the analysis done by Wegman’s team as well as known commentaries by Jolliffe, an expert on PCA. Wegman called it simply “bad science†and the mathematical treatment simply “wrongâ€. Jolliffe called it “a piece of dubious statisticsâ€.
You know better? Present the evidence.
Nope – you know precisely what I would say and I know what you would say, so I will not waste this blog’s time. I couldn’t care less about revisiting that anyway – it makes no difference to the fact of confirmation bias (in fact, if your judgment is right then it simply confirms it).
Otherwise I think you’re simply projecting your own prejudices upon someone else.
Do you? I think I am pointing out the obvious fact that self-styled ‘sceptics’ are all over ‘warming evidence’ whilst entirely having been trumpeting ‘cooling evidence’. You think that’s my prejudice? I think it’s the truth of the matter. As for my evidence, shall I do a count of the number of ‘cooling stories’ at a site like ‘WUWT’ set against the number of ‘warming stories’? If you really are not aware of the bias in the self-styled ‘skeptic community’ then I doubt we can get much further with this dialogue.
Gerald Browning,
Yep, I guess you hit it on the head. Scientists can be oh-so-clever but many often have a poor grasp of the geographical realities,
Simon, regarding Dr. Mann:
If a registered M.D. or engineer did what Mann did in the MBH9X case, he would no longer be registered. No more ‘trends’ would be allowed from him.
That he even still has a seat at the table and other scientists would collaborate with him is troubling to me.
EJ,
If a registered M.D. or engineer did what Mann did in the MBH9X case, he would no longer be registered.
I’m sure that both you and I have read copiously around this subject. I come to a different conclusion to yours. You’re entitled to your judgment as I am to mine, but I certainly don’t think you should state your conclusion as if it were a matter of fact, particularly since it implies professional misconduct on the part of this scientist. I am hoping that you will spare me the exhaustion of putting my view fully, since far too many words have been written on this subject already. I understand your grounds for a critical view, though I do not accept your judgment, and I think you should understand my grounds for a different view, though you will not accept my judgment either. I thoroughly reject your assertion that Dr Mann has been guilty of any such professional misconduct.
Re: Simon Evans (Comment#10587)
.
The reason for the interest in Dr. Steig’s paper is because he makes a claim that is a fundamental departure from anything in the literature to date; namely, that the warming in Antarctica is NOT primarily in the peninsula, but extends throughout Antarctica. Whether true or false, such claims are bound to generate interest – from skeptics, true believers, and denialists alike.
.
As far as your implication of confirmation bias, I would say that you are applying that far too generally. You might choose to note that my first post on Steig defended him against accusations that problems with the Harry AWS station may have undermined the paper. You might also choose to note that I (among other skeptics) do not trumpet “cooling evidence”. You may choose to note that I have said that I believe AGW is real – with the caveat that I believe the current model predictions to be too high.
.
Or you may choose to note none of these things.
.
For the Mann comments by others: AFA the anti-Mann rhetoric goes, I do not have anything against Dr. Mann. I’ve read the Wegman report. I’ve read the criticisms against his reconstructions; I’ve read his responses. Personal opinion aside, it is clear from the Wegman report that, though his methodology was shown to be prone to errors due to spurious correlations, he was commended for having the gumption and insight to take on the task in the first place. The Wegman report also does not say that the hockey stick is provably false; they state that Dr. Mann’s conclusions are plausible, albeit with less certainty than Dr. Mann originally claimed. The report leaves open the possibility that the conclusions are also potentially false. The fact that the report does not issue a verdict that Mann’s conclusions are demonstrably false is strong evidence, in my opinion, that Mann is certainly not guilty of any deliberate falsification or intent to deceive.
.
Besides, the fact that he has a Ph.D. (and I do not) leads me to confer some respect upon him as he at least had the dedication to finish what I did not.
.
I won’t pull any punches about what I think of the Antarctica paper, since I feel I can back those statements up with verifiable and reproducible evidence. But I don’t feel that ad hom arguments against Steig and Mann really accomplish anything except polarize people into sides, which is counterproductive.
Ryan O (Comment#10599)
Ryan,
Thanks for your response. I was not having a pop at your analyses – I apologise if it seemed that way. I’ve found what you’ve posted to be relevant and interesting, and I appreciate your work on the subject. Of course I accept that Steig et al is an interesting finding and will thus prompt responses.
No, my comments really were to do with the appropriation of the term ‘sceptic’. I don’t think you’ve made any point of claiming that for yourself, you’ve just got on and said honestly what you have to say on the basis of your study. By contrast, I expect you’ll realise that there has been much rejection of the paper by self-styled ‘sceptics’ who cheerfully admitted that they hadn’t even read it. I think it is a reasonable call to say that our observational record of the Antarctic is not very good, but we should always have been aware of that. I will note that the ‘cooling meme’ has been trumpeted by some, though I do not lay that at your door.
I note all of the points you make as to what you’ve said before, and had already noted them.
As for Mann et al 98/99, I take the view that there were statistical inadequacies in the analyses and agree that the levels of certainty were unsound. However, I have seen no evidence whatsoever to justify the view that there was any malfeasance in the work, and I think that is confirmed by the NAS review itself and by comments made by NAS reviewers.
I agree with you entirely that –
I don’t feel that ad hom arguments against Steig and Mann really accomplish anything except polarize people into sides, which is counterproductive.
Regards,
Simon
Simon,
.
Thanks for the reply. 😉
.
I may be guilty of taking a few potshots at Mann every so often . . . but I stop short of accusing him of fraud because, frankly, I don’t think he’s being fraudulent and it’s a pretty serious accusation to make.
.
And no, I don’t find any evidence of malfeasance in Steig’s paper, either. Confirmation bias maybe . . . but not fraud.
OK, but Mann’s refusal alone to archive, let alone refuse to provide data, codes and methodology would be enough to lose registration as a professional. No?
If I am not mistaken, an FOI was required. Is this acceptable professional conduct in your opinion? Does this conduct by Mann advance the science?
Simon, I hear what you say about confirmation bias, but I don’t think it’s unreasonable that new material from a controversial source is subject to more careful scrutiny. It seems to me that the “skeptics” and the “warmers” (“non-skeptics”? I’m not sure what is an inoffensive term) provide mutually beneficial service, each group looking carefully at results they find less plausible. Both camps have bigots and cheerleaders, but that’s the way the cookie crumbles. I’ve yet to see any discussion on the motivation of the various players reach a sensible conclusion.
EJ:
.
This is not engineering. Mann’s refusal may be bad sportsmanship, but his refusal to release the information (and he has released quite a bit – albeit after some prodding) makes no statement on the rigor or quality of his work. The two are unrelated.
.
.
I do not think that what he did helped advance science, but that is not even in the same universe as saying he should no longer be allowed to practice science.
Simon Evans #10592
I am not surprised that you wish to avoid actual discussion of Mann’s “hockeystick” papers. To wit: “Nope – you know precisely what I would say and I know what you would say, so I will not waste this blog’s time. I couldn’t care less about revisiting that anyway – it makes no difference to the fact of confirmation bias (in fact, if your judgment is right then it simply confirms it).”
What Wegman said:
Wegman Report
1. In most datasets, the first principalcomponent should be the least smooth (because of the higher variance). However, inMBH98, MBH99, the proxy data are incorrectly centered, which inflates the variance of certain proxies and selectively chooses those decentered proxies as the temperature reconstruction.
2.The controversy of Mann’s methods lies in that the proxies are centered on the mean of the period 1902-1995, rather than on the whole time period. This mean is, thus, actually decentered low, which will cause it to exhibit a larger variance, giving it preference for being selected as the first principal component. The net effect of this decentering using the proxy data in MBH98 and MBH99 is to produce a “hockey stick†shape. Centering the mean is a critical factor in using the principal component methodology properly.
3.Findings
In general, we found MBH98 and MBH99 to be somewhat obscure and incomplete and the criticisms of MM03/05a/05b to be valid and compelling.
Sorry for the time waste, but just how do you spin that smackdown into anything other than an utter rejection of the conclusions.
MarkR [10583]
How about IARC-JAXA? That is probably as close to the bone as you will come.
Quoting from McCullough and McKittrick’s new paper:
“claim (Mann, Bradley, and Hughes, 1998: 779) that the principal contribution of their study was their “new statistical approach†to analyze global climatic patterns.
The “new statistical approach†turned out to involve a standard method popular among applied statisticians and available in every statistical software package, called “principal components analysis†(PCA). Mann, however, wrote his own PCA code in Fortran, despite the fact that well-debugged and polished versions of the algorithm were readily available in existing statistical packages. Mann’s PCA code did not give the same answer as SAS, SPSS, or other statistical packages would have.”
And:
“Mann released an incomplete portion of his code in July 2005, seven years after his paper had been published. Among other things, the code revealed that Mann et al. had calculated the insignificant R2 statistics, but had failed to report it. Also, files found on Mann’s FTP site showed that he had re-run his analysis specifically excluding the bristlecone pine data, which yielded the result that the hockey sick shape disappeared, but this result was also not reported.
In the context of private sector due diligence, a failure to disclose adverse performance is a misrepresentation. It is no less serious in scientific contexts.”
So — we know that Mann wrote his own code instead of using standard code and his code was screwed up. Corrupt? Or just incompetent? Doesn’t matter. Either way, it doesn’t inspire trust in his work.
We also know that he ran his stuff without the verboten bristle cones and he knew he had no hockey stick without them. And we know that he knew his results were worthless by ordinary statistical measures. None of which he disclosed.
Doesn’t exactly inspire trust.
SimonE [10592]:
Maybe it is time to learn to recognize when you are severely wingshot [so to speak…]
SimonE [10598]
Try submitting a chemistry or bio sciences paper with the “materials and methods” section essentially left bare [even to your closest “peer review” friends] and see how fast it is defenestrated from the highest floor in the building .
Leaving that absolutely fundamental section of a scientific paper – bar none- empty, essential because without it, it is impossible to verify a researcher’s findings, and doing that repeatedly in the “Team’s” various “peer reviewed” papers, makes an absolute mockery of their purported “science”.
I have flagged on this blog before that I have a good number of years of both scientific and corporate due diligence experience and as my present and former colleagues will testify, have a reputation for very sensitive “bullshit detectors” [as in global temperature changes “indistinguishable from zero”].
So please, would you prove me [ and by extension the others on this blog] wrong on the very specific point you have raised, because as you no doubt understand, it goes to the very heart of science and its core percepts.
No skating, please.
RyanO “…they state that Dr. Mann’s conclusions are plausible, albeit with less certainty than Dr. Mann originally claimed.”. I believe that came from the very bad NAS report. Wegman on the other hand found the Mann et al 98 to be wrong from all angles, as was implicitly accepted by the Congressional Committee. MS. SCHAKOWSKY “..this false or inaccurate Dr. Mann study”
Peter (Comment#10606)
I have read the Wegman Report, but thank you for the quotations. Here are Dr Mann’s responses to the ‘Committee On Energy And Commerce
Subcommittee On Oversight And Investigations’, from which I will quote:
“There is another element of this question which raises a deeply troubling matter with regard to Dr. Wegman’s failure to subject his work to peer review, and Wegman’s apparent refusal to let other scientists try to replicate his work. Professor David Ritson, Emeritus Professor of Physics, Stanford University, has found error in the way that Dr. Wegman models the “persistence†of climate proxy data. Interestingly, this is the same error Steven McIntyre committed in his work, which was recently refuted in the paper by Wahl and Ammann, which was in turn vetted by Dr. Douglass Nychka, an eminent statistician. Dr. Ritson has determined that that the calculations that underlie the conclusions that Dr. Wegman advanced in his report are likely flawed. Although Dr. Ritson has been unable to reproduce, even qualitatively, the results claimed by Dr.
Wegman, he has been able to isolate the likely source of Wegman’s errors. What is so troubling is that Dr. Wegman and his co-authors have ignored repeated collegial inquiries by Dr. Ritson and apparently are refusing to provide any basic details about the calculations for the report (see Attachments 3 and 4 to this Response). It would appear that Dr. Wegman has completely failed to live up to the very standards he has publicly demanded of others.”
http://www.meteo.psu.edu/~mann/house06/HouseFollowupQuestionsMann31Aug06.pdf
Here were David Ritson’s enquiries to Wegman, to which he received no response. I quote from one:
“To facilitate a reply I attach the Auto-Correlation Function used by the M&M to generate their persistent red noise simulations for their figures shown by you in your Section 4 (this was kindly provided me by M&M on Nov 6 2004 ). The black values are the ones actually used by M&M. They derive directly from the seventy North American tree proxies, assuming the proxy values to be TREND-LESS noise.
Surely you realized that the proxies combine the signal components on which is superimposed the noise? I find it hard to believe that you would take data with obvious trends, would then directly evaluate ACFs without removing the trends, and then finally assume you had obtained results for the proxy specific noise! You will notice that the M&M inputs purport to show strong persistence out to lag-times of 350 years or beyond.
Your report makes no mention of this quite improper M&M procedure used to obtain their ACFs. Neither do you provide any specification data for your own results that you contend confirm the M&M results. Relative to your Figure 4.4 you state “One of the most compelling illustrations that M&M have produced is created by feeding red noise (AR(1) with parameter = .2 into the MBH algorithm”.
In fact they used and needed the extraordinarily high persistances contained in attatched figure to obtain their `compelling’ results.
Obviously the information requested below is essential for replication and evaluation of your committee’s results. I trust you will provide it in timely fashion.”
http://www.meteo.psu.edu/~mann/house06/RitsonWegmanRequests.pdf
So, should Wegman be accused of professional misconduct in the way that Mann has been?
I think, Peter, you are mistaken in your assumption when you say “I am not surprised that you wish to avoid actual discussion”. I have no fear of discussion. I have given my reasons above for thinking it would be tiresome, pointless and off topic to rerun the entire debate on this thread.
It’s quite amusing that in response to a remark I have made above where I suggested that “‘scepticism’ appears to me to be inclined in one direction” we now have a number of posters wanting to run through the anti-Mannian catechism yet again. It is being wielded because Mann was fourth author on the paper under discussion, and thus according to the faith the paper is suspect. That is, as I have said, self-evident confirmation bias.
I am, of course, subject to confirmation bias myself, just as any human being will tend to be. I hope I do not, however, make assertions as if they were matters of fact when I know that in truth the matter in question is the subject of dispute.
tetris (Comment#10613)
So please, would you prove me [ and by extension the others on this blog] wrong on the very specific point you have raised, because as you no doubt understand, it goes to the very heart of science and its core percepts.
No skating, please.
I can’t respond to your demand because I simply don’t know what you mean. “Wrong on the very specific point you have raised” – what is that?
Ralph Becket (Comment#10603)
Ralph,
I agree with your comment above. I think my first comment (elsewhere) on the Steig paper was that especially since it was a new approach we should treat the findings cautiously, and time would tell how well it stood up to review, replication and further research in the area.
Personally I don’t think that whether Antarctica has been net warming or net cooling over this time period tells us much on its own about the general case of AGW, although it’s obviously important to get the facts right if we can (and then to understand them).
I think it’s fair to Steig to stress that he clarified what the paper does not show in his first RC post on it, before publication – not contradicting recent East Antarctica cooling, not asserting a long-term trend, not overturning their own finding of the 1940s being the warmest decade in the 20th century. Some of the dialogue that has happened since, IMV, is best understood in the context of a long-running feud, which is not really very good for the science (that comment does not refer to Ryan O’s post above).
Anyway, I think my original wry comment was misplaced on this thread, since I realise now it could be seen as a criticism of Ryan’s approach, which it wasn’t intended to be. The thread should be about responses to his work, so I am embarrassed to have taken it off topic and will try to make this my last comment (if others would kindly not challenge me further to rerun the hockey-stick war – some other place, some other time perhaps!).
Simon,
I note the lack of any response from you on the issues of: 1) Mann’s creating bad code from scratch rather than use readily available code; 2) his use of bad data, knowing it was bad and knowing it was necessary to get the results he wanted; 3) his knowing failure to disclose that his results failed standard measures of significance.
I suspect the reason you didn’t respond is that there aren’t any good answers for why Mann did what he did.
And note, we haven’t even touched on the most ridiculous aspect of his “study” — the splicing of the temperature record to his tree ring “record” at the inflection point despite the divergence problem. That’s so bad, it would be a bizarre joke, if it hadn’t had such serious consequences.
RE M&M v. Mann one just has to go read the recent PNAS comments. Mann et al. does not answer to many of the specific criticisms…rather just a hand wave it would seem to me. Even if you don’t truly understand a topic you can tell a LOT about the validity of arguments based on how they are presented and defended, IMO. Which, BTW, is one of the reasons that Lucia has good credibility…her arguments are well thought out with ample detail about her hypothesis and assumptions and she openly discusses the arguments of those that disagree with her.
stan (Comment#10620)
Ah well, so much for hoping to end this O/T stuff – but I will respond to insinuations that I have avoided questions, so hereyago –
At a news conference at the headquarters of the National Academies, several members of the panel reviewing the study said they saw no sign that its authors had intentionally chosen data sets or methods to get a desired result.
“I saw nothing that spoke to me of any manipulation,” said one member, Peter Bloomfield, a statistics professor at North Carolina State University. He added that his impression was the study was “an honest attempt to construct a data analysis procedure.”
http://www.desmogblog.com/nrc-exonerates-hockey-stick-graph-ending-mann-hunt-by-two-canadian-skeptics
I put more weight upon the judgment of the review panel of the NAS than I do upon yours.
Addressing your specific points:
1) I am not aware that this criticism was made either by NAS or by Wegman, so I consider it not likely to be significant.
2)& 3) You have no evidence of such malfeasance. In fact, your allegation that Mann covered up the dependence of his results upon a certain proxy (bristlecone pines from the Western U.S.) is entirely bizarre when you look at what was written in MBH99:
In using the sparser dataset available over the entire millennium (Table 1), only a relatively small number of indicators are available in regions (e.g., western North America) where the primary pattern of hemispheric mean temperature variation has signi cant amplitude (see Fig. 2 in MBH98), and where regional variations appear to be closely tied to global-scale temperature variations in model-based experiments [Bradley, 1996]. These few indicators thus take on a particularly important role (in fact, as discussed below, one such indicator — PC #1 of the ITRDB data — is found to be essential), in contrast with the post AD 1400 reconstructions of MBH98 for which indicators are available in several key regions [e.g., the North American northern treeline (“NT”) dendroclimatic chronologies of Jacoby and D’Arrigo, 1989].
It is furthermore found that only one of these series — PC #1 of the ITRDB data — exhibits a signi cant correlation with the time history of the dominant temperature pattern of the 1902-1980 calibration period. Positive calibration/variance scores for the NH series cannot be obtained if this indicator is removed from the network of 12 (in contrast with post-AD 1400 reconstructions for which a variety of indicators are available which correlate against the instrumental record). Though, as discussed earlier, ITRDB PC#1 represents a vital region for resolving hemispheric temperature trends, the assumption that this relationship holds up over time nonetheless demands circumspection. Clearly, a more widespread network of quality millennial proxy climate indicators will be required for more con fident inferences.
Some cover-up! Publish the hidden truth in GRL! I do sometimes wonder what proportion of the Mann-hunters have actually read the papers in question rather than just accepting other people’s views. If you had read the above, stan, then how could you conceivably maintain your allegation? How, for that matter, can McCullough and McKitrick state that “this result was not reported” when the statement is there in black and white in the MBH paper?
I suspect the reason you didn’t respond is that there aren’t any good answers for why Mann did what he did.
I suggest you should ask people rather than favouring your suspicions. The reason I didn’t respond immediately is that I have nothing I really need to say to someone promulgating unproven allegations of professional misconduct that isn’t obvious. Mind you, it is quite fun to have demonstrated the obvious falsehood of the ‘cover up’ allegation by quoting directly from MBH99.
Also, I was waiting to track down the “McCullough and McKittrick’s new paper” that you referenced. I discover it is a study published by the Fraser Institute. May I presume that all fourteen (I think it is) case studies that the document considers are, in your judgment, the work of fraudsters, exposed as such by the very fact of being criticised by these authors?
Perhaps you should join Lucia, Myself, Monckton and Mann for duelling at Weehawken before the Heartland Conference 😉
No, Simon. Simply because the authors criticize a study is no reason for me to assume fraud. What kind of “logic” is that? And I didn’t assume anything with Mann. You’ll note that my statements are quotes from the cited paper. And, given the various professional pieces that McKitrick has published on Mann’s work, I think we can surmise that McKitrick is intimately familiar with the study.
Nothing you’ve written establishes that the paper’s summary of Mann’s work is incorrect. I’ll stand with them.
stan,
“I think we can surmise that McKitrick is intimately familiar with the study.
Nothing you’ve written establishes that the paper’s summary of Mann’s work is incorrect.”
Well, if direct quotation from MBH99 is lesser evidence than McKitrick’s ‘summary’ of it, then what else can I hope to say? Bye for now 🙂
Simon Evans,
As a point of internet etiquette, if you are the guilty party in derailing a discussion, you do not get the last word in an attempt to bring the discussion back on track. If you insist on having the last word on Mann’s work, then you are failing to live up to your stated intentions to get the discussion back on track and failing to meet your obligation under the rules of civil discourse. IMO anyhow.
Equating a lack of credibility to a confirmation bias is just plain silly and is no excuse for the derision and scary quotes you use around the terms ‘sceptic’ and ‘skeptic’. There will always be some amount of “opinion noise” in any blog discussion and the degree to which you choose to wallow in that rather than discuss the topics at hand will establish your credibility and value amongst the readership.
Earle Williams (Comment#10628)
Earle, as a point of internet etiquette, I think it is better to make points about what other posters say rather than to diverge into personal criticism.
If you insist on having the last word on Mann’s work
If you read the thread again you will see that I have effectively had demands to respond to points regarding Dr Mann even though I had made clear early on that I thought it a waste of time and O/T. I am sorry if you feel my responding to such challenges is evidence of my insistence. Personally I think that is an unfair judgment, but then I would think that.
the derision and scary quotes you use around the terms ’sceptic’ and ’skeptic’.
It is neither derision and nor is it scary. I do not accept that very many of those who term themselves ‘sceptics’ are actually sceptical by nature – certainly no more than I am. My use of quotation marks indicates that the term has been coined as a title. If you would like to suggest an alternative and have it promulgated then I would be happy to adopt it. You may note that I do not use the terms ‘denier’ or ‘denialist’, as opposed to many self-described ‘skeptics’ who use the pejorative term ‘alarmist’. In other words, I think your criticism of the use of terms is not only without substance but also very poorly directed.
There will always be some amount of “opinion noise†in any blog discussion and the degree to which you choose to wallow in that rather than discuss the topics at hand will establish your credibility and value amongst the readership.
I note that you have made no contribution to the discussion of any of the topics at hand so far on this thread but have instead chosen to introduce the topic of your disapproval of my posting style. Personally, I don’t have much respect for those who resort to the ad hominem, and think that it undermines their credibility.
Administrator note:
I’ve got a bad cold. But, popped in a moment. This is RyanO’s thread, and he’s doing a great job. But, I would like to suggest that people avoid discussing their opinions about professional misconduct on the part of Mann and/or Wegman, and focused a bit more on what RyanO is learning about what we know about warming or non-warming in Antarctica.
Ryan’s doing some pretty heavy math here. We can have food fights on other threads!
Simon Evans: Just so I have it right, are you actually saying that the methodology and conclusions of the MBH studies are valid?
jae (Comment#10633)
jae,
I think there were methodological inadequacies in the tentative (and actually quite ground-breaking) analyses of MBH ten years ago, I don’t think that the significance of such inadequacies was that great, and I do think that the ‘conclusions’ have been broadly reached by other studies since. I happen to think that we shouldn’t look for too much accuracy from any proxy studies of the kind, but that it gives us a reasonable sense of the fact that if we’re not already beyond the highest temperatures of the past 2,000 years, say, then we very soon will be if the planet continues to warm. I also think that the Mann-hunt tends to be a rather bewildering substitute for any substantial scientific discussion that offers an alternative view to the logic of AGW.
But now I fear someone will say I’m trying to have the last word, even though I’m just responding to your question 🙂
There are none so blind ….
UPDATE: Added a movie that you can run in R to see how the satellite recon temperatures and station temperatures compare over time. The stations are pretty easy to pick out. 🙂
How sensitive are the first 3 PCAs to square waves? In frequency analysis, square waves are very noisy because the sudden changes produce a lot of frequencies.
When dealing with calculating the mean, infilling with zero is neutral. But zero can be a sudden change from adjoining positive or negative values. Can one infill with an average of adjoining values, and what does that do to the PCA characteristics?
Scooter, they first use RegEM to do the infilling. RegEM starts by infilling zeros for missing values, but then replaces the zeros with values calculated from the covariance between stations. They then calculate the PCs based on the infilled data. So there shouldn’t be any sharp changes.
Lucia, did you get my e-mail via the Contact Link?