{"id":21509,"date":"2013-01-23T15:23:40","date_gmt":"2013-01-23T21:23:40","guid":{"rendered":"http:\/\/rankexploits.com\/musings\/?p=21509"},"modified":"2013-01-23T18:38:26","modified_gmt":"2013-01-24T00:38:26","slug":"a-defense-of-the-ncdc-and-of-basic-civility","status":"publish","type":"post","link":"https:\/\/rankexploits.com\/musings\/2013\/a-defense-of-the-ncdc-and-of-basic-civility\/","title":{"rendered":"A Defense of the NCDC, and of Basic Civility"},"content":{"rendered":"

There is a cancer growing in the climate blogging world. It is a cancer of bad faith, a default assumption that the other side must be lying, stupid, or in the pay of someone nefarious. It manifests itself in one-sided discourses, and personal attacks, and in the blind rejection of results that do not conform to a specific world view.<\/p>\n

I tend to be hard to rile up by nature, but a recent article on Fox News<\/a> was too egregious to be ignored. It was not the criticism of data adjustments that was the problem (though this was somewhat unfounded, as I will discuss later), but rather the remark at the end attributed to Anthony Watts. He said:<\/p>\n

\u00c2\u00a0Is history malleable? Can temperature data of the past be molded to fit a purpose? It certainly seems to be the case here, where the temperature for July 1936 reported … changes with the moment. In the business and trading world, people go to jail for such manipulations of data.<\/p><\/blockquote>\n

I\u00e2\u20ac\u2122m sorry, but accusing people that you disagree with of fraud, and even suggesting that they go to jail, is simply beyond the pale. Not only does it stymie any possibility of constructive scientific discourse; it is also blatantly unethical. Fraud should only be alleged in extreme cases when there is strong evidence supporting it, not simply because the results don\u00e2\u20ac\u2122t match your preconceptions. If you disagree with someone\u00e2\u20ac\u2122s approach and methods, the proper way to respond is to create your own approach and demonstrate that it is superior. That is the way science moves forward. To descend into personal attacks, to politicize the science, is deeply irresponsible.<\/p>\n

This type of discourse also creates immense distractions for the scientists involved. Many of the folks at the NCDC have spent a significant portion of their time over the past three years dealing with two different GSA investigations, various congressional hearings, and the need to respond to media furors like the Fox News story. This is not to say that we cannot be skeptical of the results of scientists like those at NCDC, but rather that the correct approach to that skepticism is handled through scientific arguments rather than political or media attacks.<\/p>\n

In the spirit of civility, I would ask Anthony to retract his remarks. He may well disagree with NCDC\u00e2\u20ac\u2122s approach and results, but accusing them of fraud is one step too far. Given all the steps the scientists at NCDC have taken to publish their data and code<\/a>, to make their papers accessible without a pay-wall<\/a>, and to work with external groups to evaluate and verify their findings, there is no reasonable justification for the allegation of fraudulent behavior, and certainly not to suggest the scientific work they are doing is a jailable\u00c2\u00a0offense.<\/p>\n

_______________________________________________________________<\/p>\n

Now that that is off my chest, lets look at the evidence surrounding NCDC\u00e2\u20ac\u2122s approach to homogenizing temperature data. Temperature data in the United States is imperfect. It is measured at weather stations that were not intended to create long-term climate records, and over the past century these stations have moved (e.g. from building rooftops to airports and wastewater treatment plants in the 1940s), the instruments have changed (e.g. from liquid-in-glass thermometers to electronic sensors in the 1980s), the time of day at which the temperatures were measured has changed (from evening pre-1970 to morning), and the environment around sensors has changed as cities have grown and more land has been developed.<\/p>\n

\"USHCN<\/a><\/p>\n

(Click to Embiggen)<\/p>\n

To correct for these issues, the National Climate Data Center (NCDC) at the National Oceanic and Atmospheric Administration (NOAA) has created methods to detect and remove so-called inhomogeneities (biases due to non-climate factors). This approach, called the pairwise homogenization algorithm (PHA), is an automated method that look for step-changes or spurious trends that occur at one station but are not seen at the same time in nearby surrounding stations. The idea is that climate operates on larger regional scales, and local changes not seen in the regional average are likely artifacts station moves, sensor changes, etc.<\/p>\n

The net result of these adjustments, at least for the United States, is a significant increase in century-scale temperature trends. Notably, the same adjustments don\u00e2\u20ac\u2122t have a particularly large effect world-wide, suggesting that some of the cooling biases corrected are specific to the U.S. (or, conversely, that parts rest of the world has too little station coverage to adequately detect breakpoints). The figure below shows both raw and adjusted contiguous U.S. temperatures from 1895 to 2012 (with a baseline period of 2004-2006).<\/p>\n

Given the magnitude of the adjustments (and their direction), it is important to ensure that biases associated with things like station moves and instrument changes are being properly removed without introducing any additional bias. To this end, there have been a number of different efforts, both by folks within the NCDC and external groups that test and evaluate the effectiveness of adjustments. These include tests on synthetic data, the creation of pristine reference networks (e.g. USCRN), independent analysis and homogenization methods developed by the Berkeley Earth group, and comparisons to satellite records. Lets examine each in turn.<\/p>\n

 <\/p>\n

Tests with Synthetic Data<\/h3>\n

\"williams<\/a><\/p>\n

The best way to determine if an automated algorithm is working properly is to test it using synthetic data with know biases, and see if the algorithm correctly removes those biases and produces a result in-line with the \u00e2\u20ac\u0153true\u00e2\u20ac\u009d unbiased data. To that end, the folks at NCDC worked with Peter Thorne to create eight synthetic worlds. Each of these worlds had a \u00e2\u20ac\u0153ground truth\u00e2\u20ac\u009d created by sampling the climate fields of four different global climate models at the locations of all U.S. co-op stations. Each world then had different types of bias added in. Some worlds had negative net trend biases, while others had positive ones. Some worlds were characterized by lots of small break-points, while others had large breaks. Some worlds had good metadata documenting breakpoints, while others had little to no metadata or even false metadata. It is also important to note that this was a blind evaluation; the NCDC researchers did not know which world was which or what the \u00e2\u20ac\u0153ground truth\u00e2\u20ac\u009d of each world was prior to running the algorithms, removing any temptation to tune the algorithm to produce the desired results.<\/p>\n

Overall, the PHA did quite well in this exercise, removing the majority of bias in all cases. It correctly cooled worlds. The figure below shows the results for one representative world. The raw biased data is in red, and has twice the century-scale trend of the true data, in green. The default run of NCDC algorithm is shown in yellow, with a few dozen variants of the algorithm (created by varying the breakpoint detection parameters and correction methods) shown in black. More details on this exercise are available in the Williams et al paper<\/a>\u00c2\u00a0<\/b>published last year, as well as our recent AGU poster<\/a>.<\/p>\n

\"Screen<\/a><\/p>\n

Similar work has been done by a European group<\/a> that tested the accuracy of the PHA-homogenized individual station records rather than regional temperature reconstructions, as well as other homogenization methods. It showed that the PHA was one of the better-performing methods (though not the best). There is still more work to do in the area, specifically around testing data with articial trend-biases added to individual station records rather than simple break-points, and it is quite likely that more papers will come out in this area in the next year or two. The new International Surface Temperature Initiative<\/a> led be Peter Thorne (in partnership with NCDC) is putting a large focus on creating standardized tests and metrics for homogenization algorithms, for example.<\/p>\n

 <\/p>\n

The U.S. Climate Reference Network<\/h3>\n

\"Screen<\/a><\/h3>\n

Starting in 2001, the NCDC began to deploy the U.S. Climate Reference Network, a distributed network of stations with pristine siting, uniform instrumentation, frequent readings, and multiple redundancies to provide the most accurate possible record of U.S. climate change. Unlike the existing USHCN\/co-op network, which was set up primarily to measure weather, USCRN\u00e2\u20ac\u2122s purpose is to provide a continuous and uniform measure of the U.S. climate. Since 2004 there has been enough station data to create a temperature estimate for the full conterminous U.S. temperatures which, as shown below, are largely identical to those from the USHCN network.<\/p>\n

\"Screen<\/a><\/p>\n

This is clearer when we look at monthly data since 2004. While there are a few notable differences for individual months, the trends of the two are nearly identical over the 2004-2012 period, providing further evidence that the reported USHCN data is not far off the mark.<\/p>\n

\u00c2\u00a0<\/i><\/p>\n

The Berkeley Earth Project<\/h3>\n

\"USHCN<\/a><\/i>The Berkeley Earth Surface Temperature project developed their own homogenization process involving cutting station records at detected breakpoints, recombining station fragments using a least-squares method, and creating a spatial field via a kriging approach that applies regional correlation-weights to individual station records to down-weight divergent records. They use a larger set of stations than NCDC uses, as they can effectively integrate short non-continuous station records. Their results for the conterminous U.S., however, are effectively identical the homogenized NCDC data. While it is possible that both approaches suffer from as-of-yet undiscovered methodological flaws, the fact that an independent group using a significantly different homogenization approach found identical results should serve as a powerful validation of the NCDC\u00e2\u20ac\u2122s approach.<\/p>\n

\u00c2\u00a0<\/i><\/p>\n

Satellite Data<\/h3>\n

\"uah-lt-versus-ushcn<\/a>Recent work by John Christy and Matt Menne have resulting in an interesting comparison of UAH and USHCN data for the conterminous United States over the satellite record (1979-present). While minimum temperatures are more difficult to directly compare due to the decoupling of the nocturnal boundary layer, maximum surface temperatures show a remarkable correlation with satellite lower tropospheric temperatures (r-squared = 0.96, vs. 0.86 for minimum temperatures). Notably, UAH is much closer to the homogenized data than to the unhomogenized data, providing an additional independent validation of the NCDC\u00e2\u20ac\u2122s approach. To forestall the obvious question, we would expect satellite TLT and USHCN records to be the same over the U.S., as there is no expected amplification<\/a> over land areas.<\/p>\n

The homogenization approach taken by NCDC is not perfect, and will almost certainly be improved in the future. Indeed, they just launched a new version of the code (v2.5) that is significantly better at detecting breakpoints. This is the way science works; it iterates, and responds to reasonable criticism by making improvements. What is critical to understand is that the folks at NCDC, like most scientists, are making good faith efforts to improve our understanding of the climate of the U.S. and the world. Accusations of fraud, especially when the preponderance of evidence in the literature supports NCDC\u00e2\u20ac\u2122s approach, is deeply counterproductive to the advance of science.<\/p>\n","protected":false},"excerpt":{"rendered":"

There is a cancer growing in the climate blogging world. It is a cancer of bad faith, a default assumption that the other side must be lying, stupid, or in the pay of someone nefarious. It manifests itself in one-sided discourses, and personal attacks, and in the blind rejection of results that do not conform … Continue reading A Defense of the NCDC, and of Basic Civility<\/span> →<\/span><\/a><\/p>\n","protected":false},"author":9,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[15],"tags":[],"class_list":["post-21509","post","type-post","status-publish","format-standard","hentry","category-data-comparisons"],"_links":{"self":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/posts\/21509","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/users\/9"}],"replies":[{"embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/comments?post=21509"}],"version-history":[{"count":0,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/posts\/21509\/revisions"}],"wp:attachment":[{"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/media?parent=21509"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/categories?post=21509"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rankexploits.com\/musings\/wp-json\/wp\/v2\/tags?post=21509"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}