If nothing else, the climategate emails brought skeptics questions about the reliability of the surface temperature record to the attention of the broader public. It appears that, rightly or wrongly the a sizable fraction of the broader public sympathizes with the skeptics to some extent. It’s also evident that public trust for climate scientists has diminished, as has public belief in the theory of AGW.
Because the broader public are simultaneously taxpayers, whose funds the efforts to create these sorts of records and voters, who ultimately are the ones who, through plebiscites influence how we collectively respond to the potential dangers presented by AGW, this public sympathy for skeptics means that scientists must find ways to regain the trust of the public. More importantly, they must find ways to convince the public that the findings resulting from scientific programs are trustworthy even if the scientists themselves are not.
In that light, I think it’s worth while to discuss the possibility of modifying programmatic structures for developing the surface temperature record; the ideal programmatic structure should incorporate features that permit the public as a whole to believe the product (i.e. surface temperature record) is trustworthy, and that the trust springs from knowing the process is designed to minimize the possibility that systematic errors have been overlooked as a result of human frailties to which all, including scientists, are subject. These frailties can include political bias, desire for professional advancement, tribalism or all three simultaneously.
For this reason, I am taking up Judy Curry suggestion that we explore Steve Mosher’s question, which, it seems to me, amounts to “How can ‘we’ develop a program that will create a temperature record that those totally outside the process can trust?”
Specifically, SteveMosher asked.
…Rather than rehashing the past, what would you guys suggest as a way of moving beyond this once and for all.
What entity should be in charge of the “official†global temperature index.
Some thoughts.
1 They should not be associated with anybody who runs a GCM and checks the model against the observation.2.They should spend a lot more money on the problem than GISS or CRU do.
3. Permanent staff with the right disciplines. Archivist. Legal.
Historical. professional programmers. Statisticians.4. IV&V.
5. Open source software. GPL.
6. Creative commons data.
7. Provisions for confidential data.
So like it’ll never happen, but if you had to design the way to do it, would you have problems with the above?
Just get CRU, NASA and NOAA out of this record keeping datacentric task.
With that in mind, I open the floor to suggestions to people who might provide ideas. Of course, the first question to Mosher will be “What’s IV&V.” When that’s posted, I’ll add it to the quote.
Hat Tip Steve Mosher wishes to give a hat time to SteveMcIntyre as the originator of the idea that we should focus on ways to improve the process for creating the surface temperature anomaly products.
Independent Verification and validation
Ot, give extra importance to #4 and #5 on this list.
1) The most important is to separate data gathering from data analysis. For example, NASA should gather data from satellites, but not analysis it. NOAA should analysis data, but not gather it.
2) Use satellites. Temperature stations are to labor intensive with to many possibilities for errors.
3) All data and computer code used for the entire process available to the pubic.
Re: magicjava (Feb 12 09:45),
I disagree with you. People who collect data should always be involved in analyzing it and creating final data products to at least some extent. The things that are better kept separate are testing theories and collecting data to test theories. We don’t always need a brick wall between the two. But, one of the biggest dangers in science is that the desire to confirm pet theory will introduce confirmation bias in the data.
I do think there needs to be more transparency in the data acquisition and processing. But I don’t think people deploying thermometers should be barred from creating the ultimate data record.
Re: magicjava (Feb 12 09:45),
I actually like ground based thermometers.
It might be a useful idea to incorporate an Anthony Watts idea and formalize it. Currently, some thermometers are manned by volunteers. It might be nice to find an separate set of volunteers so that each station is visited four times a year to take photos of the stations and fill out a checklist. These could be posted on at a blog, and people could see the extent to which current stations comply with quality guidelines.
The volunteers don’t need to do anything but take photos and fill out a form.
Magicjava, I think that your point 3 is truly the key one:” 3) All data and computer code used for the entire process available to the pubic.”
I’m not ready to give up on the historical data that has already been recorded. It should be possible to go back recover much of the original station records and history and then do an open transparent processing of that data.
In the next few days I will be submitting to the “The Independent Climate Change Email Review”, http://www.cce-review.org/index.php a suggestion that they test the CRU data management and analysis system by attempting to replicate the CRU data processing of a few selected stations.
In an ideal world, if I have an interest in the weather/climate at Kiribati, for example, I should be able to go to a data repository and find ALL station records for that location. I should also be able to execute standard code and see how it combines multiple station record files if they exist, perform corrections and adjustments (well documented and rationale explained, of course), and end up with the same time history that is used by the CRU for their gridded temp product.
I don’t see any reason that we cannot have that sort of transparency and ability to replicate results with respect to the time series history of existing data.
It might take several millions to a few 10’s of millions of dollars to fund a project to truly clean up and make accessible historical records and their analysis, but IMO it would be money well spent.
Lucia,
Methinks Anthony’s approach would be less practical outside the U.S., and the USCRN should deal with siting issues in the U.S. going forward (though its always worth double checking).
That said, it is prime time to update the GHCN (since it was last updated in 1997). There may also be some neat tricks we can do to use remote sensing and satellite imaging to check on urban/rural status of GHCN stations.
Magic,
NASA does collect the satellite data (AMU). UAH and RSS analyze it :-p
CRU does need to get around to releasing their data (that which they can) and code. GISS has done it for ages. UAH should if possible, and I have no clue if RSS does or not.
On a broader note, there are some questions here about priority. If spending millions improves the surface record slightly (and I imagine most of us here suspect it wouldn’t change much), is it the best use of funds? I certainly would like the money to be spent, but I can see it being a tough sell.
RE: Lucia
People who collect data should always be involved in analyzing it and creating final data products to at least some extent.The things that are better kept separate are testing theories and collecting data to test theories. We don’t always need a brick wall between the two. But, one of the biggest dangers in science is that the desire to confirm pet theory will introduce confirmation bias in the data.
Well, lucia, I think it’s the data that makes or breaks those theories. NOAA, as far as I know, doesn’t make climate models. But the curve over decades produced by their modifications to the data looks a lot like the curve over decades produced by the climate models from NCAR. In other words, NOAA’s analysis of the data results in adjustments that are nearly equal to the answer predicated by NCAR’s climate models. That’s very problematic to me.
But a way to make both of us happy would to be to clearly separate the data obtained from each step. Satellite data already does this to some extent. There’s raw transmissions from the satellite, there’s the packaging of those transmissions into a file, called Level 0 data, and so on. At each step of the process you can see exactly what data went into the process. I’ve got a note on how it’s done by the aqua satellite here: http://magicjava.blogspot.com/2010/01/aqua-satellite-data-processing.html
Assuming each “level” of data is made available to the public, I could live with a procedure like that. The only additional change is the source code used to process each level of data would also have to be made available.
But I don’t think people deploying thermometers should be barred from creating the ultimate data record.
You’re 100% correct, and I’m wrong on this one. But somehow there needs to be a way to clearly mark “questionable data” and trace it’s flow through the system. And there needs to be a way to exclude “questionable data” from processing when desired.
The raw satellite data includes various descriptors on where the data came from (what instrument, what satellite, what date, etc.) Perhaps simply accumulating the descriptor tags as the data flows through various processes would be sufficient.
In the end, we should be able to “plug and play” different data sets and different processing routines cleanly. If NOAA wants to take raw data and subtract 0.1 degrees from all of it that’s 3 decades old and add 0.6 degrees to all of it that recent, let them do that. But I should be able to take the same raw data and apply my own analysis to it with minimal effort.
Zeke–
I too suspect the surface record would not change much. There is a limit to what you can do to improve a product based on data collected over a century ago.
The issue is how to foster trust. How much is that worth? I’m not sure actually– but if people are going to act on the basis of the results, the public needs to feel the process assures the results are trust worthy.
P.S. In regards to SteveMosher suggestion to GPL climate software, I’m very much against this. There are many organizations that simply won’t use GPL due to its viral nature. Even less viral licensing like Apache or MIT have issues.
There’s always been a solution to making intellectual property available to the public. It’s called Pubic Domain.
Re: magicjava (Feb 12 10:27),
Have you considered either of these two possbilities:
1) The NCAR climate models at least kinda-sorta work, and the agreement stems from this?
2) To the extent that much of the data predates the NCAR models, the models and forcings may be tweaked to some extent?
Both of these seem more likely to me than the notion that NOAA tweaks the data to match the models. This is particularly true since the data after 2001 don’t match the models particularly well. If the experimentalists were fidding with the data to match the models, you’d expect predictions to look great!
Yes. This steps should be separate from the process of making the product. This is done in DOE’s ARM program. Meta data are collected along with data to flag possible problems. The flags are set by those collecting the data at the time data are collected.
Zeke Hausfather (Comment#33096),
“I certainly would like the money to be spent, but I can see it being a tough sell.”
I don’t think so. The amount of money is not all that great. If you wanted to install 1,000 perfectly sited, geographically representative, solar powered and fully automatic weather stations around the world satellite based data retrieval, it couldn’t cost as much a a couple of scientifically wothless flights to the “international space station”. And compared to the costs for action on CO2 emissions control (or if you prefer, the potential costs for lack of action), the cost for absolutely credible surface based weather data is a rounding error.
.
Same thing for a reasonable number (a few hundred?) of automatic weather stations floating on the ocean surface.
.
The cost is relatively small. And good data is one of the very few things everybody (skeptics to alarmists) can probably agree would be helpful. I’m all for it.
I don’t agree with the idea that we should give up on surface temperatures and just use satellites, as although the UAH records appear to have an integrity unmatched by any of the surface datasets, there is considerable uncertainty as to the precise relationship between tropospheric and surface temperature changes. The satellite record could, however, be used to establish an upper bound for surface temperature movements, always assuming that the GCMs are right in stating that the greenhouse effect means that the troposphere should heat at least as fast as the surface.
“kinda-sorta work”
Stunning. Well, not really.
Lucia, I remember you using words like this years ago.
We’re really not advancing any science here, are we?
Andrew
The Argos buoys is an example of how, at a reasonable cost, we have greatly expanded our knowledge of the ocean heat content. That multinational effort would be a good model for the development, deployment, and operation of a worldwide weather data gathering network.
The automated stations proposed by Zeke could even use the same satellite data retrieval system used by the Argos buoy network.
http://www.argos-system.org/html/system/how_it_works_en.html implies that any platform using an Argos certified transmitter can work with the existing Argos data handling system. See http://en.wikipedia.org/wiki/Argo_%28oceanography%29 for a general overview of the Argos Buoy system.
Have you considered either of these two possbilities:
1) The NCAR climate models at least kinda-sorta work, and the agreement stems from this?
2) To the extent that much of the data predates the NCAR models, the models and forcings may be tweaked to some extent?
Both of these seem more likely to me than the notion that NOAA tweaks the data to match the models. This is particularly true since the data after 2001 don’t match the models particularly well. If the experimentalists were fidding with the data to match the models, you’d expect predictions to look great!
Well, the old data isn’t changed that much. It’s the new data that’s changed. And no, I wouldn’t expect folks changing the data to make it match exactly.
But the issue isn’t NOAA and NCAR. Those were used as examples. I’ve read Dr. Karl’s papers and can say he at least has justification in the literature for using the adjustment tools he’s using.
But proper adjustment tools can be used correctly or incorrectly, just as I can take a hammer and hammer a nail or break a window. Is Dr. Karl correctly applying the adjustments he uses? I don’t think anyone can show he is or isn’t. No one knows what the source code is.
So the issue isn’t Dr. Karl, or NOAA, or NCAR, it’s can I get the data from each step? Can I get the computer code from each step? Can I audit the data and the processing done to the data? Can I swap out someone else’s adjustments with adjustments I feel to be more appropriate?
Right now, I can’t. There are roadblocks, intentional or unintentional, every step of the way.
It occurs to me that funding agencies could improve things very much by writing contract to emphasize certain deliverable rather than simply considering writing a journal article a suitable end product.
Jones might have been protected from himself if the DOE contract supporting Jones has listed the following as deliverables:
1) The source code used to create the CRU record. The contract could specify that the code should include access to all required subroutines and that once delivered, this made for hire code would be considered property of the DOE.
2) The archive of all station data used to create the surface temperature record. (This data could either be typed out or provided on tape.)
DOE could also have stated its intention to permit public access and inspection of the code as delivered. Even more ideally, the task of running the code to maintain the record and post new monthly values could have been assigned to someone outside CRU. After all, at this point, it’s not really science, is it? (CRU might get continued funding to provide guidance on dealing with new stations etc. But someone would be running the delivered code.)
Obviously, to deliver both 1 and 2, CRU would have to deal with the pesky issue of getting people to agree to make the data public and they would also need to create a stable version of the code which would, essentially, be archived. The fact that a third party would ultimately run the code would result in some external inspection– even absent formal IV&V.
When everyone above says something about the data I hope they mean the raw data. The raw data must be archived and available to all who wish to do their own analysis of it. It should include all the meta data about the site it came from including equipment upgrades, moves, etc..
Any processing/manipulation of the data must be accompanied with full and complete disclosure of the methodology and fully documented data processing program code. These items are also to be archived for posterity and public access.
These two items will insure that anyone who wishes can do an independent IV&V and publish it.
Why should there by any provision for confidential data?
We want to get away from the position where somebody can say “My secret evidence proves X or Y, I can not show you how it does that, but trust me it really does prove X or Y”.
All data in the archive should be usable and accessible by all. By all, I mean all – anybody on the planet who wants to use it.
If that means some organizations have to be persuaded, or even paid, so as to release the data, then so be it.
If a few organizations refuse to free their data, then the analysis simply has to be done without those bit of data.
David,
Why does the UAH record have “integrity unmatched”? I’m still far from convinced its better than RSS, and there are plenty of “adjustments” that go into turning MSU data into temperature estimates. In fact, the net errors found over time in the satellite records vastly exceed anything found in surface records (recall, for example, that satellite records showed cooling until an error was discovered a few years back!).
It seems that there is a tendency to trust UAH because a) Spencer is a skeptic and b) it shows a slightly lower trend than other records. One could argue that some folks prefer GISS for the opposite reasons, but the other temp series (CRU, RSS) agree more with GISS than UAH.
Lucia,
“Both of these seem more likely to me than the notion that NOAA tweaks the data to match the models. This is particularly true since the data after 2001 don’t match the models particularly well. If the experimentalists were fidding with the data to match the models, you’d expect predictions to look great!”
.
Well sure, but substantial manipulation of the recent data is just about impossible, since RSS and UAH trends are available to “verify”, at least to a reasonable degree, that the ground based trends are accurate. More credible are worries about adjustments to older data, and I can at least imagine that the adjustments on older data made by GISS and Jones could be subject to a bias to make the GCM’s look better (I’m not suggesting that is the case, only that it is possible). The only way to dispel doubts is to make certain that all the unadjusted data, all meta data, and software source code are available to anybody who wants them.
Re: Stephen Singer (Feb 12 10:59),
In my comment discussing deliverables, I meant raw station data should have been delivered. The program should be those required to take raw stations data and turn it into the final product. If this involves a series of programs, so be it.
Use station data from high-quality rural locations only.
Use only data from stations with a long record and no station moves.
Use raw data only and apply no adjustments to the data.
[quote Stephen Singer (Comment#33117)]
When everyone above says something about the data I hope they mean the raw data.
[/quote]
I mean all the data. Raw data as well as data that’s been processed. Each step of the way in the pipeline, the data used should be made available to the pubic. There’s an example of how the Aqua satellite does this here: http://magicjava.blogspot.com/…..ssing.html
The only problem with the above example is that while all the data from each step is carefully kept separate, not all of it is made public.
You also need the computer source code for each processing step. “Black box” verification of the processing isn’t good enough.
I owe an acknowledgement to Steve McIntyre for this whole question. He suggested long ago that the temperature index be turned over to separate body. he suggested the MET. My spin on this is rather to tease out the principles rather than point out a specific body. So, lucia, can you add a hat tip to SteveMc, cause I stole the idea from him.
“3. Permanent staff with the right disciplines. Archivist. Legal.
Historical. professional programmers. Statisticians.”
Would it be good to substitute “Actuaries” for “Statisticians”? I’ve never encountered a statistician but I occasionally hear of people who make their living as actuaries. As I understand there are societies of actuaries who administer tough examinations to credential people as competent to render opinions on risk and uncertainty. I think I could put more faith into someone who earns a living in the insurance field than an academic whose life is governed by the goals of getting published and getting tenured.
Re: Zeke Hausfather (Feb 12 11:02),
UAH is without question better than RSS since 2002 because UAH uses data from the Aqua satellite, which has station keeping capability, and RSS doesn’t. Aqua data does not need to be corrected for orbital drift. Any correction will always degrade precision and may not actually improve accuracy. QED.
Magicjava.
I will tell you the problem I have with Public Domain.
Its a very simple case that jeffId and others struggled with.
researcher A publishes his code for doing say “Regem” Public domain.
Researcher B takes this code modifies it, publishes a paper.
In his SI he says that he used a modified version of A’s code.
You ask B for his code. he says ” I used A’s code, its public go get it yourself”
And the fun fight begins.
I pick GPL because I know it and it works. But lets just say this.
If you use data from this center to publish a paper or sell a service or whatever, if you add value to this data by adjusting it or homogenizing it, whatever, YOU OWE your version of the data
back. You also owe your code back. basically if you use this open source data and open source code to do anything you owe the modifications back licenced under the same terms as you licenced them.
The idea of specified ‘deliverables’ for publicaly funded research is a good one, and was certainly included in every engineering or technical consulting contract I have ever entered into. I can tell you from personal experience that required deliverables get your attention when you sign the contract, and help with ‘focus’ during the contract. The nice thing about it is that it doesn’t require any new laws. It is a purely administrative issue, and could be enacted tomorrow by any public funding agency.
.
But I fear the screams from publicly funded scientists could be deafening.
Zeke:
It is a reasonable expectation that the satellite measurements should come in a bit lower than the trend from ground based, principally because the latter are in the atmospheric boundary layer, whereas the former principally measure the temperature above it.
This type of trend amplification is plentiful in climate science, compare sea to land, equatorial land to northern polar land for example.
I distrust time series that seem too like other time series considering the difference in what is being measured as well as differing measurement uncertainties and systematics involved.
Here’s an example for land-record, crutemp3v.
The possibility certainly exists that measurements within the boundary layer should show amplification relative to those directly above it.
PaulM
In my mind demanding only rural stations etc etc, is putting the cart before the horse ( define rural)
A better requirements would be that metadata should be provided openly and quality controlled. How one defines rural is a METHODOLOGICAL question. here we are only dealing with the principles and requirements of a data center that will restore trust.
Well, How is the trust broken.
1. Conflict of interest.
2. Lack of transparency.
3. Lack of independent Oversight.
4. Key missing Skills ( records managment, legal etc)
5. PR agendas ( announcing stupid warmest year ever stuff)
So I tend to view these things this was. How can I build a trusted machine.
Steven Mosher:
Moreover it shows a lack of understanding about what you are trying to measure, which is the mean surface temperature anomaly field.
It is a real impact of human activity that Europe is warmer than it would have been had the cities not been there. We aren’t trying to remove the effects of human activity, simply not over sample them relative to rural sites.
[quote steven mosher (Comment#33134)]
You ask B for his code. he says †I used A’s code, its public go get it yourselfâ€
And the fun fight begins.
[/quote]
I think that’s a valid criticism of public domain.
I believe the answer is that so long as the “major players”, NOAA, NASA, etc. are using public domain than we have an open, understood, and tested, base of code.
If “some guy” wants to publish a paper using a special process that he doesn’t share with anyone, let him. Scientists should be allowed to do those things and may even be required to do those things.
But the vast majority of knowledge would be available to the public. And if “some guy” wants to argue that his secret process is correct, and the known processes are wrong, he’ll have do show why. And that’ll be difficult to do when his process is unknown. And probably impossible to do if he wants to keep his secret process a secret even after he’s done explaining it.
Anyway, I’m not trying to be a stickler on “public domain”. Perhaps a new “scientific license” could be created to specifically address the often competing needs of the public and researchers. But I don’t think GPL fits the bill.
Zeke Hausfather (Comment#33121)-UAH is more consistent with radiosondes, and several comparisons that have been done suggest that UAH is more reliable. Read any of like a dozen papers by Christy.
The similarity to surface temps is irrelevant since, though you never seem to get it through that hard skull of yours, they aren’t the same thing!
I would like to see more ground surface temperature stations. Rather than dissuade me from the practicality of surface station thermometers, WUWT has piqued my interest in what can be learned from temperature changes due to urbanization and land use. If there are enough stations it may be possible to begin to quantify the changes and associate them with encroaching surface development. How far do these effects radiate over time?
The more pieces of the puzzle we have to work with the better.
Lucia and others suggest that “the temperature record wouldn’t change much but the two key points are
1. There are now sufficient examples of suspect “inappropriate data manipulation” to suggest that may not be true
2. The whole point of a new independent record is to remove or at least reduce the doubt in peoples minds.
Jeff ID and others have shown that around half (or may be more) the increase in global temperature record is the result of adjustments to the record and at least some of these ( e.g GISS use of coastal stations to estimate arctic sea/ice temperature ) are odd to say the least
Andrew_FL (Comment#33148),
“though you never seem to get it through that hard skull of yours”
.
Sure wish you had left this part out. Gratuitous insults serve no purpose; better to let Michael Tobis be in charge of those.
magicjava.
Agreed. Specifying a specific license is too narrow.
( the case I was talking about BTW was Steig)
The criteria should be that they do a better job, not how much money they spend. Anybody can spend more money without doing a good job. There are indications that the NOAA isn’t doing a good job at managing their stations and data, however much money they’re spending.
“The criteria should be that they do a better job, not how much money they spend. Anybody can spend more money without doing a good job. There are indications that the NOAA isn’t doing a good job at managing their stations and data, however much money they’re spending.”
Quite, AnonyMoose…
And who is going to get them to do a better job…Lucia? Steven Mosher? Judith “Lip Service” Curry? Mommy? Bugs, Boris or Nathan? The Borg Collective?
Andrew
I apparently posted the wrong link to the Aqua satellite data processing pipeline. Sorry about that. Here’s the correct link:
http://magicjava.blogspot.com/2010/01/aqua-satellite-data-processing.html
I fail to see why this data is different from any other statistics collected by national government statistics departments. They already employ specialists in statistics, archiving and data presentation. They have no axe to grind in having the statistics prove something. They already collect data from a wide ranges of sources and have international agreements for sharing it, standardising it and publishing it. Most of the source data comes from other government agencies anyway.
Why not use the existing resource and expertise?
steven mosher,
Your comments on the merits of GPL and like licenses are informative but are rather idealistic. Not that there’s anything wrong with that!
Anyone can use software licensed under the GPL without obligation to open source their work product. Anyone can modify software code licensed under the GPL and use it internally with no obligation to open source their modifications or work product. Many organizations do this.
Where the GPL comes into play is when someone wishes to distribute a GPL-licensed product or something derived from the GPL-licensed product. In that case the distribution is subject to copyright laws. the GPL is the license to distribute. Without that license the distributor is at risk of being in copyright violation.
So, to sum up, nothing in the GPL would compel Steig or Mann to release modifications they may make to GPL-licensed code, unless those individuals are distributing their modified code. The values of the open source community may help shame said researchers into releasing their code, but available evidence suggests that certain of these individuals have no shame.
I think there would be innumerable benefits to having climate science practitioners adopt open source development of their code. But neither the GPL nor any alternative open surce license will force open that which does not wish to be open.
Re: Alan Wilkinson (Comment#33165)
There is merit in the proposal but I’m not sure how it resolves the issue because people also believe that the Govt understates actual inflation in order to pay less in Social Security benefits.
You need both QA and QC functions. Sign off that sites are to standards. Periodic site inspections with calibrated thermometers. Benchmark checks to see if anybody is drifting off of the baseline. No changes of site configuration without a recalibration period to verify the old and new configuration data is consistent. No implementation of new changes such as whitewash to latex without testing to verify that there is no effect.
Not gonna happen.
RB, so long as there is proper record keeping, documentation and transparency that issue resolves itself. None of those existed at HADCRUT. All of them would be required of a Statistics Department.
Re: HankHenry (Feb 12 11:12), Sorry to disillusion you, Hank, but I have worked in the insurance industry for 30 years and run across a lot of actuaries, and they are like any other group of people: the best of them are outstanding, but by and large their mathematical methods are pretty crude compared with the kind of statistics my old university friends are doing as professional statisticians, and quite a few of them have a tendency to try to get to the answer the client wants, which is to some extent how we got here in the first place.
BTW Zeke you know much more than I do about the subject, but unlike some of the others it doesn’t look as though UAH have been caught rigging stuff.
Lucia [33106]
Anyone who has worked in a position of trust knows and will confirm unembelished that once that trust has been broken for whatever reason [once is enough, let alone multiple times as in the case of the IPCC, CRU, GISS, etc.] that person or organization or by extension those seen as representing the organization, are toast.
Broken is broken and perception is reality. No ifs, buts and whats. It takes years to build a reputation of trust and it takes only a few minutes to break that beyond repair. I have seen investors put millions of dollars into a venture based on trust in its management, and CEOs fired in one board meeting for breaking the trust.
Hard as this may be for some of the “troopers” to accept, reality is that the collective “climate science institution”, represented by the likes of the IPCC, CRU, UK Met Office, GISS, NOAA, a slew of otherwise respectable universities as well as those governments pushing the policies hardest, in the eyes of the “great unwashed” is discredited beyond redemption.
Mike Mann building an academic empire around hockey stick science that never got rid of its odour? And his university that gets millions in R&D grants and overhead from that enteprise, and is then supposed to conduct an “independent” inquiry into his activities? The voters on the street understand self interest and conflict of interest way better than the elites give them credit for.
This means that whether AGW/ACC is real or not [totally irrelevant at this point], no politician [subject to the votes of the “great unwashed”] is going to risk his/her seat on the basis of the new “perception is reality”. If anyone reading this post in the US has doubts about that observation, just ask your nearest “blue dog” Democrat.
Trust onced broken can’t be repaired. Ever. And in the case of “man-made global warming”, with the broken trust goes the entire meme and narrative.
SteveF (Comment#33154)-The problem is that I have admonished Zeke endless times not to directly compare satellite to surface temps. He never ceases to think he is doing nothing wrong it doing so. It is maddening as hell.
Re: magicjava (Feb 12 09:45),
“1) The most important is to separate data gathering from data analysis. For example, NASA should gather data from satellites, but not analysis it. NOAA should analysis data, but not gather it.”
This pretty much happens now. CRU and GISS don’t measure temps. NOAA updates GHCN from met sources. UAH doesn’t manage satellites.
Re: David (Feb 12 14:48),
“unlike some of the others it doesn’t look as though UAH have been caught rigging stuff.”
No “others” have been caught rigging stuff.
Here is a thought. Before you discount it, contemplate. All temperature monitoring equipment should be placed at local golf courses. The monitoring equipment is basically inocuous. That would end complaints about the surounding black-top and concrete that skews data. They still need to rate rural vs city or urban because of the larger umbrella of UHI. But I as golfer know the golf course would automatically qualify the site as well placed in 99.9% of the cases, it would be hard to place it badly to skew the reading. If you would google the golf courses in all the developed world you would find them nearly everywhere. Sincerely, John P
Really interesting thread. If i can take a step back to get some context and generalize what i am seeing here in terms of what is wrong with the system.
If i can generalize from these posts, the group of skeptics represented by the technical blogs (e.g. CA, Lucia) which also comprises the people that made the FOIA requests of CRU, want data quality control, accountability, and generally a data set that can be trusted. There is no apparent motivation related to nuking proposed carbon mitigation policies or seeking revenge on climate scientists or distracting them from their work.
General problems contributing to the lack of trust in the surface temperature data set are:
1. inadvertent introduction of bias (the Michael Crichton argument) if the people who create the data sets are also using the data sets (e.g. modeling groups and research scientist who use the data to test hypotheses). Note NOAA does have one of the 3 climate modeling groups in the country, GFDL but GFDL has little interaction with NCDC. At NASA GISS, the same group is creating the dataset and using it to verify climate models. There is also a clear link between CRU and the UK Hadley Centre model.
2. Apparent agenda (and lack of objectivity) from the groups preparing these analyses in terms announcing warmest years/months, predicting future warmest years/months, and in some cases explicit advocacy activities
3. Lack of transparency, in terms of not clearly being able to link raw data to the final analyzed product. Sufficient info should be available for the analyzed data set to reproduced by an independent group. (I also have a bone to pick re the confidential data. what kind of commercial value could this possibly have? aren’t these countries getting tons of free weather/climate info from U.S., EU, Japan in terms of satellite data and weather climate predictions? Surely the UN/WMO could work this out)
4. Creation of the surface data sets is treated like a research project, not a contract with deliverables, including data quality management, records management, response to user queries, etc.
5. Lack of independent oversight, such as data quality analysis and an Inspector General legal type to make sure concerns are adequately addressed and responded to.
Am i missing any other big picture problems along these lines?
Thanks Dr. C,
I’ll just speak for my self ( since I don’t take direction very well )
“If i can generalize from these posts, the group of skeptics represented by the technical blogs (e.g. CA, Lucia) which also comprises the people that made the FOIA requests of CRU,”
1. Some people are skeptics, Lucia and I believe it is getting warmer as a result of adding GHGs to the amosphere. Some of us have issued FOIA to CRU. Some for data, others for agreements after CRU gave conflicting responses to data requests.
“[you] want data quality control, accountability, and generally a data set that can be trusted. There is no apparent motivation related to nuking proposed carbon mitigation policies or seeking revenge on climate scientists or distracting them from their work.”
Carbon mitigation: on the table, both Lucia and I support a strong move toward nuclear. I’ve supported Ross McKittricks T3 Tax.
Revenge and distraction? Hardly, in the past some have suggested schemes to use FOIA to distract. It would be transparent. Not a place we are willing to go.
“General problems contributing to the lack of trust in the surface temperature data set are:
1. inadvertent introduction of bias (the Michael Crichton argument) if the people who create the data sets are also using the data sets (e.g. modeling groups and research scientist who use the data to test hypotheses). Note NOAA does have one of the 3 climate modeling groups in the country, GFDL but GFDL has little interaction with NCDC. At NASA GISS, the same group is creating the dataset and using it to verify climate models. There is also a clear link between CRU and the UK Hadley Centre model.”
Generally speaking I think you want to remove all appearences of potential influence. For my own part I cant recall if flight test reported into engineering or directly into the program office. The point is to have some structure to prevent modelling groups from influencing data collection and analysis groups. For example, you dont want people controlling GHCN ( say Peterson) being paid out of GISS coffers.
“2. Apparent agenda (and lack of objectivity) from the groups preparing these analyses in terms announcing warmest years/months, predicting future warmest years/months, and in some cases explicit advocacy activities”
Yes. Its probably best if they just announce the facts with out
positioning statements. Its .65C. Advocacy groups can very easily spin to their hearts content. But NOAA ( for example) should just hew to the bare facts.
“3. Lack of transparency, in terms of not clearly being able to link raw data to the final analyzed product. Sufficient info should be available for the analyzed data set to reproduced by an independent group. (I also have a bone to pick re the confidential data. what kind of commercial value could this possibly have? aren’t these countries getting tons of free weather/climate info from U.S., EU, Japan in terms of satellite data and weather climate predictions? Surely the UN/WMO could work this out)”
Yes. Its a record keeping job. Very little “science” just rather who did you get the data from, what did you do it? All as open source. WMO would be a good choice for example. But you need accountability. Legal accountability.
“4. Creation of the surface data sets is treated like a research project, not a contract with deliverables, including data quality management, records management, response to user queries, etc.”
Yup.
5. Lack of independent oversight, such as data quality analysis and an Inspector General legal type to make sure concerns are adequately addressed and responded to.
Am i missing any other big picture problems along these lines?
That’s all of it.
Also involved is ongoing improvements to the system.
Preservation of the historical record. Additional historcal research. Improved deployment of high quality measurement systems. Especially if years from now you get to regional forecasts etc
One thing I think we need to make clear is no matter what we do the algorithms used to ‘fix’ errors in the historical data are basically a scientific judgement call and no matter how hard people try they *will* prefer the algorithms that given them the result they would like to see.
What I would like to see is an ‘ensemble’ of correction/extrapolation algorithms using the same raw data as an input. The results from different algorithms could be plotted on the same graph to provide an envelop that should include the unknowable reality.
Forget about this notion of a single answer. It is not possible with the data we got.
I’m somewhat surprised by the lack of discussion regarding geographic siting and the related issue: do we really need an average global temperature? It appears that the need for latter led in part to the current grid cell / infill procedure that strikes many as a handy process for temperature chefs but not a good way to create an understandable, valid and reliable statistic. Do we need a global average or just a good way to measure temperature change over time? I’m not sure we need an average global temperature when the change in the average of all stations temperature might be our real concern. For an average station temperature, station siting and coverage becomes an important factor. If you get the siting right, we could probably accept the average of all stations as a good proxy for average global temperature and a subset of stations could become the standard proxy for regionals. Not much need to make up data with this kind of measurement system.
Related to the above, perhaps a perfect temperature measuring system could be designed to better study a couple of recurring issues, such as urban heat island and elevation effects. Off the top of my head, I’d think some kind of station pairing process (clearly urban and nearby clearly rural, with good meta data for both) might help us better interpret the historical temperature data. The same might be true for elevation.
Finallly, my experience designing data collection processes (admittedly survey rather than instrumental data) inevitable includes the much-too-late, dope slap to the forehead stage where the well hidden problem suddenly becomes glaringly obvious. Using the right design procedures that in turn help ensure the design of a good temperature measuring process could help. I’d think the good people from NASA (the rocket science folks perhaps rather than the environmentalists) must have procedures to reduce the amount of post-implementation forehead smacking.
RogerH
You’ll never get an independent body that everyone thinks is independent. And that body will probably get infiltrated by people with an agenda if you announce that this body will henceforth be the authority (certainly most here would think that of the IPCC).
It also depends if you are dealing with the past, present or future. Are you exploring the proxy reconstructions? The thermometer record? The satellite data? This is a massive task.
I think the best thing to do is just get lots of people to get the raw data and just run with it. See how well they do, and publish their results and if they do better, or come up with a more reliable reconstruction or whatever, we all benefit. If other people want to make a proxy reconstruction, go ahead and do it. If people want to examine UHI, go ahead and do it. I think it’s a bad idea to set up a neutral authority, as invariably people won;t think it’s neutral.. Unless the neutral authority is simply a data repository.
I gues I am voting for a raw data repository.
To my mind, the crucial missing piece of the ground station work is still calibration. The current ‘best ruler’ is the satellite coverage – purely on the spatial dispersion front.
.
Not “correlation,” not “error determination,” but “figure out exactly how to get my available rulers to read the property of interest both as accurately and as precisely as they can physically can.”
.
The current approaches end up comparing aggregate quantities during the satellite overlap period. But the crucial part involves extrapolating those methods to periods in which comparisons become sparser and less reliable. But the extrapolation period is exactly where we’re most interested in the ground station measurements! And the adjustments are currently applied in a fashion to minimize the differences during the overlap period.
.
The part that’s tricky to recognize is that a perfectly situated ‘surface station’ is at best a proxy measurement for the actual variable of interest when you move into using it for climactic research. Using temperature anomalies adds to the list of assumptions, while providing modest release from some systemic error.
.
If you have a thermometer that consistently reads 1.74C cooler than the satellite’s best estimate of the gridcell’s temperature across the entire record – excellent. That thermometer appears consistent, should have a usable temperature anomaly, isn’t experiencing significant UHI (or is already maxxed out), etc. This would be a straight-forward ‘offset only’ calibration – and would yield a direct measurement of the error associated with extrapolating this particular instrument to the measurement of the gridcell.
.
That is: Many of the assumptions inherent in the various ground station adjustments should be able to be fairly rigorously evaluated on a case-by-case basis.
.
Picture a microsite issue:
There’s a barbecue ridiculously close. Otherwise the site has no urban heat island effect and would be a good proxy. If there were sufficient raw data, it could be possible to pick out the days on which the barbecue was fired up. That’s quite unlikely. What should be pretty easily detectable though is both the warm bias -and- a solid estimate of the increased error.
.
Picture an urban heat island effect:
A small town has been growing over the past thirty years. The town is small relative to the size of the gridcell, so a measurement anywhere inside the town (including a large park) provides a disproportionate warming trend compared to what’s going on for most of the gridcell. In this case, direct comparison of the satellite’s estimate for the gridcell’s temperature and the ground station should show a clear trend. Once the trend is estimated and subtracted, the station would be a far more useful predictor for the gridcell temperature.
.
This is a fundamentally different approach to evaluating the surface stations. It isn’t minimizing issues and hoping for the best – but rather directly evaluating the accuracy and precision of the individual stations. IOW: The best way to avoid ‘garbage out’ is to minimize the amount of ‘garbage in’ in the first place. Attempting to do so on a purely statistical basis after-the-fact would seem to require more conformity to the initial assumptions than appears to be present.
.
Doing anything along these lines requires access to the data prior to it all being turned into monthly or yearly reports though. At least, I don’t know of any way to request satellite temperature contour plots for arbitrary historical dates and locations.
[quote Nathan (Comment#33220)]
I gues I am voting for a raw data repository.
[/quote]
Putting the data under version control would probably be a good idea too. Believe it or not, published temperature records change a lot.
Check out these graphs of the data stability for the major data sets. Lets bars mean more stability and less changes to already published data. Basically, less bars are better, more bars are worse.
UAH stability:
http://www.climate4you.com/GlobalTemperatures.htm#Temporal%20stability%20of%20global%20air%20temperature%20estimates
RSS stability:
http://www.climate4you.com/images/MSU%20RSS%20MaturityDiagramSince20080508.gif
HadCRUT3 stability:
http://www.climate4you.com/images/HadCRUT3%20MaturityDiagramSince20080225.gif
NCDC stability:
http://www.climate4you.com/images/NCDC%20MaturityDiagramSince20080517.gif
GISS stability:
http://www.climate4you.com/images/GISS%20MaturityDiagramSince20080517.gif
Given that most data sets are unstable and subject to revision, I’d like to be able to label a version and always be able to retrieve that version.
Dr Curry as to your #3, remember you’re dealing with bureaucracies (what is not explicitly allowed is forbidden). There doesn’t have to be a valid reason for confidentiality. No bureaucrat worth his swivel chair would take the chance that there might be someone who would criticize they’re not holding that data confidential. CYA (cover your agency) all the way.
One other issue that has to be addressed is the independence of the group doing the QA/QC function. Too many agencies (FAA, AEC) are charged with both safety enforcement and promotion of their respective areas of responsibilities. It creates a conflict of interest that is dangerous. In this case, the siting/collection agency must be separate from the monitoring/auditing function.
Probably a stupid idea, but looking to the future, why not encourage schools worldwide to become involved in a new and standardized global project. Many already operate weather stations as part of their curriculum. They all have websites that raw data could be entered upon, with details about site location, situation etc.
Those interested in analysis could use a web bot to download individual, regional or global raw data, accessible to everyone.
Take it a stage further, the data could be continually transmitted in real time to the schools network, and made available on-line, with timezone and lat/long embedded. You’re not then just limited to min/max analysis. This isn’t complex technology here, if we can connect millions of webcams to the net, thermometers won’t be a problem.
The data wouldn’t be owned by anyone, it would be free to statistically torture however you want, and would be provided by people with no axe to grind.
Any attempt to sort out the current systems, what with dropped/moved/poorly sited stations, closed databases, secret adjustments, and all the other problems associated with a network that evolved but was not planned, is like trying to untangle spaghetti with your todger.
Just a thought.
Raven.
The point is with a Open database and open code people get to see adjustment code.
The paradigm I’m thinking of works like this.
1. you use the data in a paper you must
A. share back ANY data or results that depend on the data.
B. Share any code used to process the data.
Haha. That’s a functional way to force openness into other areas.
Roger:
“I’m somewhat surprised by the lack of discussion regarding geographic siting and the related issue: do we really need an average global temperature? It appears that the need for latter led in part to the current grid cell / infill procedure that strikes many as a handy process for temperature chefs but not a good way to create an understandable, valid and reliable statistic. Do we need a global average or just a good way to measure temperature change over time? I’m not sure we need an average global temperature when the change in the average of all stations temperature might be our real concern. For an average station temperature, station siting and coverage becomes an important factor.”
You are specifying a solution. The goal of this exercise is to specify an organization that would come up with recommendations like the one you propose. Presumably if you staff the unit wit people knowledgable in stats and geostats and climate studies and instrument issues they will specify good solutions. Outside independent audit is a safegurad against BAU.
David- “Sorry to disillusion you…”
Thanks for the perspective. Should a business be started based on the NOT crude methods your old university friends are doing as professional statisticians? Or is it that the actuary’s skill set is unsuited to the task? I’m not trying to be argumentative, I am truly curious about what actuaries do. I get the impression from things I read in the press that competitiveness in the insurance business is diminished due to exemptions from antitrust law.
I am also troubled by this tenure thing and what people in the university set will do to get it (as well as what they will do if they don’t get it – witness UAHuntsville). I have yet to hear what it was that moved M. Mann from UVA to Penn State and I am very curious about what that was all about. It’s probably too indecent for a guy like A. Revkin of the NYTimes to ask Mann if he moved because he had reason to worry about his tenure prospects. Revkin should do a stint at the Enquirer. He doesn’t seem to have much people-sense. Furthermore the Enquirer deserves a pulitzer for what it flushed out on Edwards…. there was so much punk in John Edwards that that story is still smoking two years after the Enquirer stoked it up.
Mosh,
Even with an open system it is the climate science insiders are the ones who decide which algorithms are used to create the official datasets that are compared to the models and they will pick the ones that give them they answers they like (i.e. the ones that show they were right all along).
It is not enough for sceptics to show that the algorithm has a warm bias because we are dealing with nothing but a scientific judgement call and the insideres will always be able to come up with excuses on why their preferred algorithm is more accurate (e.g. GISS claims extrapolating into the polar regions gives a more accurate result).
That is why I am saying the official record should be an envelope that puts upper and lower bounds on the plausible temperatures rather than a single line reported as ‘reality’.
The most important functions are simply excellent record keeping, organisation and documentation, archiving and publishing.
It is not the prime function of this service to interpret the data. It is their job to create the best possible measurement quality and coverage, describe the measurements accurately, then collect, collate, archive and publish it efficiently and reliably.
It is vital that the raw data is open and fully documented to anyone wishing to explore alternative methods for interpreting it.
The data set should be as comprehensive as possible both spatially and temporally because we cannot know what future uses it may be required for. Almost certainly more detailed examination of regional effects will be wanted. As many historic record sets should be preserved and documented as possible.
Nathan [33220]
I fully agree with your final conclusion. Problem is, that should have been done back in 1988 and my guess is we would not have overarching breakdown of trust that permeates everything since Climategate [and subsequent “gates”].
The establishment pretending that Humpty Dumpty can be put back together again is sorry delusion. Politically speaking, the entire AGW/ACC issue is quietly being moved into the long grass. Just listen to Obama decoupling cap-and-trade from “green technologies” as he did in NH last week. Or France looking for a way of finessing shelving its proposed “taxe carbonne”.
An raw data respository open to all comers would have been just the thing. Just never happened back then.
Lucia, here is my answer, which really isn’t mine, but it is Richard Wakefield’s.
http://www.youtube.com/watch?v=2OjPJnEtfUE
Great discussion. To emphasize a couple of points touched on by others:
First, we need to separate the issue of creating a reliable instrumental infrastructure (which will yield reliable data in the future, such as CRN) from the problem of extracting trustworthy information from the historical records we have. The second problem is the most serious.
Second, we don’t need, for purposes of estimating climate change, a “global average temperature.” We need to know how temps have trended over the last century or so, not necessarily their absolute values. One approach might be to obtain the *decadal trends* for all stations in which there is a homogeneous record for at least a decade, average the trends for all station/decades in a each grid cell, then splice them together to get a long-term trend for that cell — without worrying about the position of the zero line.
Another approach would be a sampling method — the trends revealed by those stations with long-term, unbroken records which do not require adjustments, or where adjustments are clearly warranted by the metadata. If there are enough of those, and they are reasonably well-distributed globally (and also by altitude), they should be representative, and a check on the previous method, even if there are not many of them. Even 100, if well-distributed, would be informative.
Let’s just hope the truly *raw* data from most stations is still available, and has not been manipulated.
Steve Mosher,
“You are specifying a solution. The goal of this exercise is to specify an organization that would come up with recommendations like the one you propose.”
I disagree. It doesn’t much matter who the organization is. It could be several organizations. All they need to do is serve as a repository for data, where unadulterated raw data is archived and where met stations around the world deliver continuing data. Then anyone who wants it can obtain it, process it and interpret it as they see fit, and argue for their methods.
Here is a thought. Before you discount it, contemplate. All temperature monitoring equipment should be placed at local golf courses.
many golf courses are irrigated. this would be similar bias, just to the other side.
Re: sod (Feb 13 06:43),
The other problems with the golf course idea is that moving the thermometers is itself a bad idea and golf courses don’t necessarily remain in existence forever. The one behind my dad’s condo was ripped up. I’m sure others have been too.
It’s good to brainstorm things like that too. Unfortunately, unless creating the most perfect possible 200 year long temperature record is the most important task of mankind, we have to recognize that individual stations can be in adequate locations, with most shifted as little as possible. Some redundancy can help detect problem in individual measurements or ameliorate discontinuities during known shifts. But that’s about it.
Should the same requirements apply to all fields like for example oil fields surveys, predicted environmental impacts, economic projections, social sciences, earthquake models, sun storms modelling, disease carrier spreding models, etc.? Should all of these pass a sort of public scrutiny looking for an implausible unanimity?
If one thinks that the selfregulating rules of science does not work, it’s there that we should make changes. Trying to pull out the responsibility from the scientific community makes more harm than good.
And beside this, who should be responsible to decide that there actually is an issue? A handful of scientists in disagreement? Or worse public opinion on each and every proclamed issue “measured” by polls? Not at all clear where this process would lead us.
Re: Ahab (Feb 13 08:34),
No one is looking for unanimity. Some of the fields you describe already are subject to a large amount of public scrutiny, but the goal is not to achieve unanimity, but to achieve transparancy. Companies preparing environmental impact statements aren’t permitted to respond: “We performed an environmental impact study. It shows things will be fine. Period. That’s all folks. Move along. We aren’t going to show you what we did.”
Quite a bit of documentation is provided to oversight agencies, and also available to the public. Some member of the public who access the documents may disagree with conclusions of both the company and oversight board– but they are permitted to read the arguments, see data etc.
The goal is transparency not unanimity.
The self regulating rules of science works for somethings and not others. It can also be slow.
In any case, we currently seem to have a problem that some scientists are losing the public trust. I don’t see why members of the public can’t discuss what changes might improve public trust. Whether scientists like it or not, if someone wanst to know what makes me, (or any member of the public trust) you or the scientific process as a whole, you need to ask me — or members of the public. Clearly, you (or scientists), can’t dictate what will make me trust you and vice versa.
You may wish that I would just trust you; you may think the procedures you like should make me trust you. But if they don’t fill my requirements for trusting a third party, I won’t.
Mind you– you may also grumble that I don’t do things to gain your trust. Ok…. but if we are using this as a metaphor, the individual members of the public have little to lose if if a small group of scientists doesn’t trust them. So, few individuals are likely to change their behavior just to gain the trust of climate scientists.
Ahab (Comment#33279),
“Should all of these pass a sort of public scrutiny looking for an implausible unanimity?”
In almost every case, no.
.
But climate science in 2010 represents a unique set of circumstances which make public scrutiny inevitable. While a disease carrier model is certainly important, its importance to the general public pales in comparison to climate science, if only because of the scale of public expenditures which might be proposed based on the predictions made by scientists in those respective fields. Whether accurate or not, there is a perception held by a substantial portion of the public that climate science is practiced mainly by people who share a common set of priorities and political beliefs which lead to exaggerated estimates of future climate change, exaggerated estimates of future risks/consequences, and draconian policy prescriptions.
.
It does not matter if you personally do not think climate science has been hijacked or tilted badly by people who hold rather extreme ‘green’ and ‘left-wing’ political views. Enough of the public do believe this to guarantee that climate science will be subject to rather extreme public scrutiny. In spite of vocal protests by climate scientists, this is not going to change any time soon.
In the end, the surface temperature anomaly really doesn’t tell us all that much. That’s because the heat capacity of air isn’t a universal constant. It varies strongly with humidity and somewhat with temperature. Since we can’t measure the radiative imbalance from space with sufficient precision yet, we need to look at heat content rather than just temperature. That’s a direct measure of the radiative balance. The ARGO system to measure ocean heat content is where effort should be concentrated and we should be comparing AOGCM predictions of OHC to that rather than to the global average near surface air temperature.
lucia,
i didn’t want to say that we should not discuss this. Before any new rule on how data have to be managed can be applied it must be determine if there’s the need. Who’s in charge?
Mine was just a question on how far reaching these new rules should be and where what i think is an over-regulation would lead us.
Re: Ahab (Feb 13 10:44),
Questions like yours are ones we are discussing. Different people have different opinions– this always happens. Also, it may be that many people would be content with some subset of the rules, and would be content even if their favorite solution was not included.
I would be content with a system where funding agencies focused on deliverables and then made those available to the public. The deliverables for a program that creates something like CRUTemp should include things like
a) the archive of raw data used to create a final product like CRUTemp
b) the archive of intermediate data– like the adjusted temperatures c) the source code that transformed the raw data to the intermediate data,
d) the source code that transformed the intermediate data to the final surface temperature record.
e) a document describing any and all functions, subroutines and libraries required to run the code. Any code for these should be available to the public at a cost that could not be deemed “exhorbitant”. (It’s fine with me if it’s something like Matlab, EXCEL etc. It simply has to be a product members of the public could purchase at some remotely reasonable fee. It can’t be some internally created subroutine only available to scientists at Livermore — if they own it, they should include that in the public distribution of the code. Also, simple functions should not be tucked away inside a full package that costs $100,000 to license used by only 3 people on the planet.)
Contracts will stipulate all of the above will be made publicly accessible– and the responsibility for that should be enforced by the funding agency (e.g. DOE). It should not be left to journals, who, after all, have no obligation to enforce DOE’s rules or contracts.
If this were done, I wouldn’t worry too much about the program requiring independent verification and validation (IV&V). The transparency would assure that a volunteer army of outsiders would download the code and see what they get– and they could also actually check that the code does what the scientists journal articles claim it does.
This system doesn’t really create greater regulation. In principle, all PI’s must fulfill their obligations under their contracts. It’s just that some program managers at the DOE (or similar funding agencies) have tended to want to write many of these contracts with language that fits the notion of a “grant”– which gives a scientists money to do open ended science, and the proof of success is a journal article. The scientist has very few actual obligations to fulfill.
In contrast, other program managers do focus on deliverable which can be identified and described before the research is done and independent of any journal articles that might be published. The scientists still does the research the proposed to obtain the funds, they still write scholarly articles. But at the end, they must also deliver certain concrete products to the funding agency.- These products might include a computer program that does “x”, a data archive containing “y”, a program report that describes what the program does quite specifically, (which would be much more detailed than any journal article would even permit anyone to submit)
That said, there are others who really have an issue with the lack of a formal IV&V process. That would create a larger regulatory structure. I’d prefer the “insist on deliverables” approach to the “IV&V” approach. But.. I’m not queen of the universe. There are other collections of things that I think could also ensure the product is trustworthy.
DeWitt Payne (Comment#33293),
“The ARGO system to measure ocean heat content is where effort should be concentrated and we should be comparing AOGCM predictions of OHC to that rather than to the global average near surface air temperature.”
OHC certainly provides important insight about net energy balance and should be used to validate/invalidate many AOGCM predictions. ARGO represents an enormous advance in accuracy of measurement of OHC, but I think it is not getting as much attention as it might have, since ARGO has indicated little if any energy imbalance from 2003 to 2009. When surface temperatures stopped rising rapidly, within a few years, so did OHC, suggesting a relatively short ocean lag constant and relatively low sensitivity (a la Schwartz). Perhaps the climate science community doubts the ARGO results.
.
But I think measuring surface temperature remains important. Heat balance may not tell us everything we would like to know about surface temperature changes. It seems reasonable that surface temperatures over land may very well change significantly more than the global heat balance would suggest; e.g. arctic surface temperature could rise quite a lot and cause ecosystem disruptions, even with a small change in heat balance.
SteveF–
I agree with you. Also, testing surface temperature predictions and ocean heat content is not an “either/or” proposition. Both can be done; both present challenges.
The difficulty with using ARGO data the program is relatively new. I’m not entirely sure the people doing the measurements are absolutely sure they’ve knocked out all the kinks; even if they were, we don’t have long time series to go on.
So… I don’t do comparisons yet. That should not prevent Dan Hughes from doing those comparisons if he thinks they are interesting and has both the time and resources to do it.
i have just one point to add to this topic:
please can we all agree, that the same mechanisms (independent team collecting all information. all raw data available to everyone. free source codes of all programs involved) will also be applied to all forms of nuclear technology.
Sod–
Uhmm… A lot of these things are required of nuclear technology already. If they weren’t, we couldn’t expect the public to trust the claims and products, and the oversight groups would be unable to determine whether safety issues have been investigated with due care.
lucia (Comment#33303),
Yes, there could be kinks to still be worked out with ARGO. There appears to be some disagreement about if the OHC, ocean mass increase, and sea level are consistent with each other. I suspect that so long as OHC continues to show little thermal imbalance, there will be considerable reluctance in climate science to accept the ARGO results. (Just like there is reluctance to accept that the IPCC model predictions are inconsistent with the surface temperature data, and reluctance to accept that model predictions of tropospheric temperature profiles are inconsistent with model predictions.) WRT a long ARGO record, it would be good to have a record covering many years, but since OHC represents, at least in theory, a low noise integration of the (noisy) instantaneous heat balance, ARGO data should be reasonably useful even with a relatively short record… if the data don’t have serious systemic errors.
sod (Comment#33304),
.
I’ll go out on a limb and guess that you: a) do want to drastically reduce CO2 emissions ASAP, but b) strenuously object to using nuclear power to help achieve reduced CO2 emissions.
.
Do I have it about right?
“There may also be some neat tricks we can do to use remote sensing and satellite imaging to check on urban/rural status of GHCN stations.”
Only if the images are good enough to actually SEE the station. I have seen a number of urban sites classed as rural because of minor coordinate errors combined with satellite “nighlight” data. We must remember that weather stations were basically set up for weather prediction, nobody ever envisaged that accuracy to 0.01 degree/year would be needed or wanted (and it is probably unachievable, in any case).
Leo G (Comment#33265), it is true that the average temperature is going up primarily due to an increasing temperature of the lows (usually night time temperature).
As I understand it, the argument is reasonably simple, increase the CO2 content, means the temperature near the surface increases. This warmer temperature means more moisture gets “baked” out of the soil, resulting in a higher mean nighttime humidity. This in turn resulting in less radiative heat loss at night, resulting in warmer nocturnal temperatures.
This result is also very easy to observe if you compare temperature swings in tropical regions (which tend to have high humidity levels) to more temperate ones (which tend to have lower humidity levels). Higher relative humidity almost always translates into a smaller swing from maximum daytime to minimum nighttime temperature.
Shorter version, warming the minimum without warming the maximum temperatures still implies a warming trend.
Hat Tip Steve Mosher wishes to give a hat time to SteveMcIntyre as the originator of the idea that we should focus on ways to improve the process for creating the surface temperature anomaly products.
Are you trying to make me cry? What utter bollocks. McIntyre is not the originator of the idea that we should improve the process for creating the surface temperature anomoly products. Focus on it? One of the problems of climate science is that it is so complex, and Trenberth has already said the main problem at the moment is trying to account for the energy movements within the system. Focussing on the surface temperature to improve what is already reasonable accurate is only going to refine our understanding of one small component of the system, which has greater needs at the moment.
Sod,
If you want to compare it to nuclear technology you should acquaint yourself with the protocals in that industry. You don’t know what you are talking about and you dont make coherent arguments.
The argument on the table is this: Constructing a independent agency or program to collect and maintain raw temperature observations is a good step to creating more trust in the quality of the record.
You can argue against that proposition is the following rational ways.
1. Its not needed.
2. Its not practical.
3. Trust will never be re established.
1. It’s not needed. You could argue that the current records and proceedures are good enough. Problem: if your goal is to convince people who doubt the record, you have not answered their objection. In sales we call this closing on objection. You identify the objection ( see dr. curry) you CONFIRM that it is an objection ( did you see her do that, yes) then you close on the objection.
2. You could argue that it’ll never happen. Defeatist argument not based on emprical evidence. You try to close on the objection.
3. Another defeatist argument. basically the tactic would be to include Key skeptics ( say Christy 4 example)on some sort of review board. If you worked in industry’s were these kind of lack of trust things happened you would understand.
This aspect of climate science, however, is not overwhelmingly complex.
.
And yet the confirmation studies of the part that is quite complex are compared with the “instrumental record” – that is, the part that has all the problematic issues.
.
The historical temperature records are a sizable piece of the foundation of the topic. If the global average temperature estimate for 1900 was discovered to be off by 2C – in either direction – it would involve quite a bit of rethinking, retuning and adjusting on the part of the modelers.
Bugs.
You will have to forgive me. In school I learned that whenever I got an idea from somebody else I should give them credit.
Lucia and Dr. Curry credited me with the idea of a independent agency handling this data. It wasnt MY idea. I read it first from Steve Mc. Other people may have had the same idea, the idea may have been around all over the place. I speak for myself. I first heard this idea from STEVE. So I hat tip him. I’ll give you another example. “noble cause corruption” One day when he and I were chatting (about how tired we were and how depressing it was to read all the mails) I said, something to the effect of this:
It boggles my mind that they would go to all these lengths to cover up something ( proxy nonsense and CRUTEMP) I mean the underlying problems are not that great. And steve said, he too was somewhat puzzled and threw out the notion of “noble cause corruption” I liked metaphor and asked him if I could use it. And when I do, I credit him. It was the same with the term “lukewarmer” I credited Bender and he came on to correct me that it was someone else. Jeez dood.
Have you never written a paper Bugs?
Good idea Lucia.
Many good ideas in the comments too.
One thing that I would add, is how important it is that the temperature records program be a well defined written down process that includes configuration management. That way, it will not be so individual people dependent, especially if it includes built-in independent review requirements. Some of this has already been mentioned in the above comments for example magicjava suggests version control. I suggest both the process, which must be written, and the database be under a formal change control system. Changes should be approved by previously determined organizations, including independent ones. Documentation giving the rationale for the change with approval signatures must provide a record for each and every change.
A formal data and software quality assurance program should be a part of the program as well.
I agree with the others sod, that these requirements and much more are already a part of Nuclear Technology, implemented decades ago. (I worked in the nuclear industry for most of my career.)
steven mosher (Comment#33344) February 13th, 2010 at 3:57 pm
Do you honestly think McIntyre was the first person to think it would be a good idea to improve the temperature record?
“Do you honestly think McIntyre was the first person to think it would be a good idea to improve the temperature record?”
When something so simple flies over his head, I wonder why you people even take the time to answer him.
Re: the suggestions above.
transparency is the only recourse. The people in charge now have, not might have, lost the trust of a large part of the public. I have many friends who still don’t understand the warming trend we have right now and with all the revelations coming out are digging in with renewed enthusiasm. It seems redundant but this all should have been done ages ago. When M&M did their trick, the cat was out of the bag and it’s always harder to put it back in.
Rant over. Keep up the good work Lucia.
Re: TGSG (Feb 13 21:00),
When something so simple flies over his head, I wonder why you people even take the time to answer him.
No, it’s a good question. Do you?
GHCN was set up in the early ’90’s, exactly with that purpose.
I’ll just add that the National Snow And Ice Data Center looks like a pretty good prototype for this to me.
*) Data is easy to access.
*) Source code is available.
*) Discussion on the history and technicalities of the algorithms and instruments are available.
*) They don’t actually gather data.
The only thing I would add to their setup is version control for the data and code that would let me pull down previous versions.
Zeke: There may also be some neat tricks we can do to use remote sensing and satellite imaging to check on urban/rural status of GHCN stations.
I was thinking about UHI on a walk tonight.
I did a comparison of GHCN raw with and without ‘high population density’ stations. (no sig difference) But that is only one indicator of urban. And I don’t know how accurate the gridded data I used is (GRUMP). One thought I had was to multiply the pop/km^2 by GDP per capita to get $/km^2 as an indicator of ‘urbanity’. But per capita GDP can vary greatly across regions in a big country like the USA, Brazil, China, and Russia.
The satellite ‘night-brightness’ is an attempt to identify urban stations from space. But there have to be other methods: Infrared? Spectral images delineating vegetation from pavement and buildings?
Jeff Id has an interesting post up on combining surface and satellite data:
http://noconsensus.wordpress.com/2010/02/13/7686/
Just a quick correction:
I used GWP, not GRUMP, in my analysis.
GRUMP includes “urban extant” data as well as pop density and other data as well.
Guess I have an update to do.
http://sedac.ciesin.columbia.edu/gpw/global.jsp
Carrick, thanx. Got it. Just don’t like the anomly way of expressing temps. Prefer the “real” numbers. 🙂
bugs (Comment#33359) February 13th, 2010 at 6:05 pm
.
“Do you honestly think McIntyre was the first person to think it would be a good idea to improve the temperature record?”
Bugs Are you terminally incapable of reading and comprehending. I never said he was the FIRST. Watch my lips.
1. I said I thought of it after reading what he wrote. If you want to trace it back somewhere else DO SO. But I only know what I know. I FIRST read it from him.
2. Its NOT THE IDEA OF IMPROVING THE RECORD. it was the idea
of a separate new independent AGENCY.
please read
Leo:
Actually there’s a reasonable theory behind using anomalies rather than absolute numbers.
If you use absolute numbers, you have to accurately calibrate the instruments for microclimate effects of the site selection.
If you subtract a baseline temperature, you remove much of this calibration (to first order the temperature trend is unaffected by the microsite temperature offset).
Ron Broberg,
Thermal imaging is an interesting idea, though you might have to calculate some sort of region-based thermal anomaly, since the absolute thermal value would depend on a lot of factors.
As far as incorporating GDP into urbanity, I’d be careful. In the U.S., for example, the exurbs will have a much higher income (and lower UHI) than inner-city neighborhoods.
I think Google Maps has a solid enough read (in the US at least) that one should be able to determine a pseudo population density from nearly live data.
That is, the metric “Number of street addresses within a half-kilometer” should be pretty good at picking out both true wilderness sites and urban sites, with a reasonable spread as one moves through smaller towns and then villages.
Interesting link to a UHI review via ClimateProgress
http://www3.interscience.wiley.com/cgi-bin/fulltext/123222296/HTMLSTART
For urbanity you have the following possible products.
some from columbia ( I think somebody cited it above) Pointed this out back in 2007
1. Population data:
2. Vegetative index ( Hansen WAS going to use this. Gallo’s work I think)
3. Impervious surfaces ( satillite product)
4. Nightlights.
the biggest issue is getting historical information. The original GHCN metadata files had datafields for decadal population figures but it looks like they abandoned the approach.
What should big oil/coal’s role be in climate science?
Fossil fuel is the foundation of our societies wealth, and a substantial portion of many shareholders individual wealth. With the fossil fuel industry under such a strong attack from AGW I would consider it highly unethical if big oil/coal did not invest the funds necessary to investigate climate science and do some independant fact checking.
So if they did do such independant fact checking what was the result?
Perhaps they did, and their fact checking confirmed AGW and they’ve been quiet and stayed out of the debate?
Or they did confirm AGW and they decided to dishonestly attempt to subvert public opinion anyway?
Or they did, found the facts wrong, decided that they would have no credibility opposing the false facts of AGW openly and have been contesting the facts from behind the scenes?
Michael Hauber (Comment#33459)
“So if they did do such independant fact checking what was the result?”
It’s all the same data.
Before this AGW stuff, before the US Oil Companies and the US military were turned into the “bad guys” of the world by the press and some citizens of this country (USA) and others (I am generalizing here); those two entities (along with other branches of the USA’s government) spent more money, developed more technology; funded more research and hired more people to clean up and take care of the environment/planet/air then any other entity, government, country or company on the entire planet. (ie; WWII made a mess). I am sure these facts still stand. And by the way; the idea of creating new green jobs is a fantasy. IMHO All the green jobs are already created otherwise global companies (like URS for example); and others (like the one my husband works for) would be hiring like crazy. They hire people from all over the world right now ; but not “like crazy” if the hype of the green jobs was true. 🙂
On topic; placing a monitoring device out in space on another planet would be good (pointed at us) out of the orbit; to monitor our planet’s temperature, incoming radiation AND orbital wobbles would be a good idea. But our President and our Congress isn’t much into funding space programs are they?
Michael H,
I see two problems with having the oil and coal industries providing research for public policy on the causes of climate change. First is the obvious bias any industry has to its own existence and second is public perception of the reults. We often hear the complaint, “skeptics are paid by the coal/gas/oil indutry”. And there is legitimate concern when (or if) it is true.
An industry will certainly makes its case and pay researchers to support them but verification must be independent. I recall the paper vs. cloth diaper environmental debate. Consumer Reports provided a scientific study showing paper is more environmental than cloth with their review of paper diapers. All my friends accepted the study as it was written in Consumer Reports and justified using paper diapers over cloth. Problem was the study was done by Proctor and Gamble who make paper diapers. Would the public be so willing to accept the results if they knew the source?
If it were possible to produce a climate model with low sensitivity, it would be a very cost effective step for the fossil fuel interests to do so. I know for a fact that at least one oil company had in house expertise, running CCM to do paleoclimate simulations (Chevron-Texaco) so they wouldn’t even have had to start from scratch if they didn’t want to.
I believe the nonexistence of such models constitutes support of the consensus position.
Michael Tobis (Comment#33755) February 16th, 2010 at 2:10 pm
“I believe the nonexistence of such models constitutes support of the consensus position.”
You can’t be serious MT. You can’t have any knowledge of the existence or non-existence, especially the latter, of all the software in use within the entire world-wide fossil fuels / hydrocarbon industries.
Michael Tobis,
.
It is easy to produce a model with low sensitivity. All you have to do what AGW climate modellers do: make up a bunch of numbers for aerosols and cloud cover that ensure the models produce the desired sensitivity.
.
The problem is such a model would be immediately rejected because of the groupthink within the climate science field.
.
Here is one such example of a simple model that reproduces the recent temperatures with a sensitivity of 1.5C per doubling:
http://www.ferdinand-engelbeen.be/klimaat/oxford.html
The idea of a central agency being responsible for the data is not right, they just need to put it online. You need a sponsor identified for all records and they are responsible for it and the sponsor signs the raw data. Signing is important as it imposes version control. The key to quality is that the collector will normally be the sponsor and they see their raw data online with a credit/photo. So if a Russian see his station is missing – or “adjustedâ€, he can add or correct it provided he is willing to be identified through signing. Where multiple folks depositing grey versions of the same data, a committee can look at the differences, and add their conclusions as metadata, but publishing all versions. We would probably need some QA scripts to look for plagiarism/copies/fake data, and any suspicions can be added as QA warning in the metadata.
If you can not find anyone willing to take responsibility as a sponsor then you do not have data, you just have a rumour. When a sponsor dies, a new foster sponsor is only needed if a new version is needed e.g. typing errors identified, or missing days found. Otherwise, the current collector signs his data as he collects it.
You impose the rule, the data is freely available, but you need to deposit your code, including station selection criteria if you publish using it. Clearly there will be a certain known how /skill needed to select which records are used, but if everyone knows what was done it will be easily to test for robustness. Cherrypicking or unjustfied adjustements will be easily caught.
To have record holders adopt the new system, a public campaign is needed. It does not require government support or funding. It needs a big database which may or may not be distributed across stakeholders, and agreement about supported data formats, and compulsory fields.
The above still allow professions to publish, and select may methods for processing the data they see fit. It just fills the requirement that most journals claim to support.
Re: cross-checking of surface data
I haven’t seen any mention of the rise of personal weather stations (PWS), familiar to any regular user of Weather Underground:
http://www.wunderground.com/weatherstation/index.asp
— especially for under-represented areas, such as
http://www.wunderground.com/weatherstation/ListStations.asp?selectedCountry=South+Africa
http://www.wunderground.com/weatherstation/ListStations.asp?selectedCountry=Chile
It might be worth spending a little public money to calibrate and evaluate these stations — or even to subsidize them — a weather station in every G8 embassy in Africa?
Cheers — Pete Tillman