Free the CRU data!
Steve Mosher popped into comments to make this request:
Free the CRU data!
Lucia, I am hijacking your thread. sorry in advance.
head over here
http://www.climateaudit.org/?p=6623And please make a request to CRU under FOI to free the data.
last time we did this we got the IPCC to post the comments on the AR4.Anybody can do this. You don’t need to be a british citizen or a scientist.
The data belongs to all of us, but only if we show them we care.
I’m not in the group who thinks there is anything super-neaky going on with the CRU data product. However, I do think greater transparency regarding the CRU data products would benefit the entire climate-wars debate. In particular, I think if the UK Met Office publicizes that temperature product and the IPCC relies on the product when assessing issue surrounding anthropogenic climate change, the broader audience for IPCC reports should be able to inspect the methods used to compute the product. By broader audience I mean: anyone, anywhere who has a right to vote or who will be affected by policy decisions by their elected officials.
For this reason, I endorse the notion that people — especially those who are distrubed by the secrecy surrounding the product– should pester the UK Met Office with FOIs.
I don’t know precisely how one goes about making an FOI request to obtain the data, but some of you may want to look into it and do so.
What I suggest you discuss in this thread
- Explain the mechanics of making an FOI request. (How to word, where to file the request, etc.)
- Discuss how to word requests.
- Discuss possible financial issues. (Might someone have to pay any fees for work to fulfill the request?)
- Unless there is a very good reason to suggest a particular person file a request, don’t suggest someone else (for example SteveM) file the request worded in the way you consider perfect, file it yourself. If you file a request, let us know.
- Note to academics: The wording of one FOI refusal suggests data would be available to academics. So, if you are an academic, maybe you could file. After wards you can collaborate with others investigating whatever aspect interests you in your research.
Comments Closed: If you would like them re-opened, Contact Lucia


Comments
James Barlow (Comment#16690) July 25th, 2009 at 10:31 am
There’s no particular trick to doing FoI. I do quite a few and in general a simple email like this is enough:
— simple foi template —
Hello,
Pursuant to the Freedom of Information (FOI) Act 2000, I’d like to obtain a copy (electronic if possible) of the following information pertaining to {Public body} :
[Insert list of requested information]
My reference for this request is (unique reference}
I would be grateful if, referring to S.16 of the FoI Act 2000, you could offer assistance with phrasing this request for information if you ascertain that a response cannot be made in its current form.
Regards,
[Your address and contact details]
— simple foi template —
Wait twenty working days, then start nagging them if they don’t respond.
Avon and Somerset Police post their received FoI requests and responses at the link below. Reading through these may help to phrase a request to be more likely to receive a response.
Examples of FoI requests
tetris (Comment#16691) July 25th, 2009 at 11:12 am
Lucia,
As I see it, the crux of this matter is the HADs harebrained attempt at evasive action by arguing that the information can only be made available to “academics”. In plain English this obfuscation means that if you were to request it under an FOI in your capacity as the Blackboard’s host you would be denied, but if you requested it under your research/academic affiliation, it would be made available to you. SteveM doesn’t have that option.
It’s the same poppycock argument that people like Plimer and Lindzen and a growing number of other eminently qualified scientists have no voice in the AGW/ACC chapter because they are not “climatologists”…..
And as for something fishy going on: there is never smoke without a fire of some sort.
Andrew_FL (Comment#16692) July 25th, 2009 at 11:16 am
Limit one per customer
lucia (Comment#16693) July 25th, 2009 at 11:23 am
Henry–
So if you know an academic, encourage them to submit. I don’t wor at a University. I am employed part-time at a national lab — not a univrsity-. I have no way of knowing how the UK met office defines that. I suspect I am also not an “academic”. In anycase, I am certainly not funded on a project that needs this data, so it’s difficult to suggest that I would be requesting in order to support any sort of “academic” research. So,when I make requests for this sort of stuff, I make them as a private citizen.
Maybe there is fire; maybe not. But the only thing to do about that reason is to get an academic to request. I suspect SteveM will ask Ross to do the FOI. But it can’t hurt for you to find someone in Canada who might request it.
Jonathan (Comment#16694) July 25th, 2009 at 12:39 pm
Ross has already made an FOI request, stating clearly that he is an academic. I have also submitted a request, worded identically to Steve’s, from my university email account. We shall see what happens.
David L. Hagen (Comment#16696) July 25th, 2009 at 12:40 pm
Stephen McIntyre’s request to Mr. Palmer:
The Met Office, UK
2008 global temperature
See: Freedom of Information, the Met Office, UK
Requests for information may be submitted to:
Marion Archer, Met Office marion.archer@metoffice.uk.gov
Global Temperature Record, Phil Jones
Climatic Research Unit
School of Environmental Sciences
Faculty of Science
University of East Anglia
Norwich NR4 7TJ, UK
Tel: +44-1603-592722/Fax: +44-1603-507784
E-mail: cru@uea.ac.uk
FOI Contact: David Palmer, Information Policy and Compliance Manager, University of East Anglia, david.palmer@uea.ac.uk
The BBC is publishing global temperature data and trends by the Met Office – without notifying the public that they do not comply with scientific principles of independent data evaluation. Readers may wish to bring this to the attention of:
Sir Michael Lyons, Chairman, The BBC Trust, BBC Complaints
US citizens may wish to formally write to:
Sir Nigel Elton Sheinwald KCMG, Ambassador
British Embassy
3100 Massachusetts Avenue, NW
Washington DC 20008
Phn: (1)(202) 588 6500
Ambassador.Sheinwald@fco.gov.uk
Scooter (Comment#16697) July 25th, 2009 at 1:02 pm
Americans, remember that FOI refers to the UK FOI act, not the U.S. FOIA. FOI details are widely available on the net, as many UK institutions include FOI sections on their web sites.
John F. Pittman (Comment#16701) July 25th, 2009 at 2:07 pm
Lucia, it is not necessarily cut and dried per “academic.” An excert from the FOI I requested:
If your position is that release of information cannot be made then, I further request a EIR/FOI request in respect to any confidentiality agreements) restricting transmission of CRUTEM data to academics contrary to the guidelines specifically listed in EuroGOOS EG99.37 Version 3, Date: 16.2.00 Policy and practice for EuroGOOS for the exchange of oceanographic and related data and products including guidelines on relationships in commercial oceanographic activities which states “Members should provide to the research and education communities, for their non-commercial activities, free and unrestricted access to all data and products exchanged under the auspices of WMO with the understanding that their commercial activities are subject to the same conditions identified in Adopts (2) above”:
1. the date of any establishment of confidentiality agreements that negate the EuroGOOS EG99.37 Version 3, Date: 16.2.00, specifically “unrestricted access”;
2. the parties to such confidentiality agreement, including the full name of any organization;
3. a copy of the section of the confidentiality agreement that prevents or negates EuroGOOS EG99.37 Version 3, Date: 16.2.00, specifically “unrestricted access”;
4. a copy of the entire confidentiality agreement that prevents or negates EuroGOOS EG99.37 Version 3, Date: 16.2.00, specifically “unrestricted access”.
lucia (Comment#16703) July 25th, 2009 at 2:23 pm
Scooter– Yes. US FOI is different from UK FOI. I don’t use either much, but I’ve never used UK FOI. So, I don’t have any idea how that works.
Steve McIntyre (Comment#16712) July 25th, 2009 at 8:47 pm
Lucia, unlike many readers, I do not surmise that Jones’ obstruction is anything other than prima donna behavior.
Like all efforts to keep climate data secret, this is an unwinnable battle for Jones and the Met Office; I do not understand why they even make the attempt.
In my opinion, the “Community” of climate scientists should be the ones writing to Jones asking him to ensure that his data is “open and transparent”, rather than standing by while Jones acts immaturely.
lucia (Comment#16714) July 25th, 2009 at 8:57 pm
SteveM–
I’ve always suspected that like me, you don’t think this odd behavior in the part of Jones means there actually is something to hide. There are soooo many other plausible theories about what is motivating Jones. (Prima-donna behavior. Pique. Territoriality etc.)
In the long run, I think Jones can’t win this. I agree with you that the climate community should be telling him to make the data transparent. Unfortunately, each individual sees no particular reason to get involved. (This is pretty common in academia. Having no opinion on the matter is the safest course.)
steven mosher (Comment#16720) July 26th, 2009 at 1:04 am
Jones is just being a prima donna; but now any error that anyone finds will be viewed in a different light.
John F. Pittman (Comment#16724) July 26th, 2009 at 6:09 am
sm #16720
That is perhaps the saddest and most frustrating part. One would rationally suppose that with the demand of teamwork by the world by the AGW proponents that anything, especailly something easily done, that could help make selling AGW easier would be done. Refusal can only lead to better stalling by the very people they wish to minimalize in the upcoming debates by giving these stallers better ammunition. Already on the internet persons are suscribing to dire theories of misbehavior. This motiff will be picked up up, and polished, and used, perhaps most effectively.
For a bit of politics:
There is at least one US Senator who has shown good capability in this regard. http://www.scientificamerican......limate-bil is a good quick article. I couldn’t find the article pointing out that the House version was passed mainly by urban districts, yet every senator has to worry about agriculture lobbyists and whiplash. A potentially interesting battle make-up for those interested in such.
MikeN (Comment#16726) July 26th, 2009 at 7:07 am
The solution to climate change is to find a wealthy guy who is against the AGW agenda, who can be a proper villain in the style of James Bond or Captain Planet. He threatens that unless the government requires open data and code for all climate research, he is going to release 1000 tons of SF6 per day into the atmosphere from his secret island lair. This is more than the US total emissions.
tetris (Comment#16728) July 26th, 2009 at 9:25 am
Steve [16712] and Lucia [16714]
I know we don’t see things quite the same way. Fact remains that IPCC stands for “intergovernmental”, etc., etc., and that for over 20 years the AGW/ACC story line has been driven by governments and not the private sector. In both the GISS and Hadley cases we have government agencies refusing to make available core data and calculations. When a private sector actor engages in this type of behaviour, all hell justifiably, breaks loose often with far reaching legal and penal consequences, as evidenced by e.g. the VIOX case in pharma or ENRON. What is particularly puzzling to me is why so many people are inclined to cut government agencies all sorts of slack in similar cases.
Quite apart from the fact that their behaviour is antithetical to the scientific method, given their track record of systematic stonewalling or throwing up smoke screens, to argue nothing is prima facia amiss and that Jones, Hansen, et. al. are merely acting out prima donna urges strikes me as singularly naive. Why impede verification when you have nothing to hide?
Andrew_KY (Comment#16729) July 26th, 2009 at 9:49 am
Using The Warmer Method of Knowledge Acquisition, I count the behavior of Jones, Hansen, et. al. as evidence that AGW is a fraud, therefore, I KNOW AGW is a fraud.
Andrew
michel (Comment#16730) July 26th, 2009 at 10:16 am
Until we eventually get the data, its all speculation, but unlike Steve M, I think the evidence doesn’t suggest prima donna behavior.
For Steve to be right we have to suppose the following. A bunch of scientists advance a controversial hypothesis. It is seriously important – if they are right, the future of humanity is at risk. They have data in lots of independent studies by different members of the group which they regard as proving the truth of this hypothesis, at least being evidence very strongly supporting it.
Yet they refuse to make it available, and cite all kinds of grounds, none of which have any fit with the importance of the issue. Like, if the future of humanity is at stake, what comparable importance can be attached to restricting release to certified academics?
It is just not plausible. People just do not behave this way. If they have real evidence for an important hypothesis, they find a way to produce it. They actually want it out in the open, it increases acceptance. Its a way of making their case. Refusal is a way of undermining it. Dixon Doll, for instance, did not say that he had decisive evidence for the smoking/lmortality connexion, but refuse to release it for 20+ years.
The only circumstances in which people behave in the way they are currently doing is if the evidence is not as represented. It has to be the default hypothesis. One case could be prima donna behavior. The pattern as we have seen it cannot plausibly be that.
Well, we will see. But remember the Chinese stations whose names could not be revealed, and their supposedly detailed metadata. What was the metadata like when we finally got the names and could check the records?
Quite so.
Andrew_KY (Comment#16731) July 26th, 2009 at 10:25 am
michel,
That was a very good post!
Andrew
lucia (Comment#16734) July 26th, 2009 at 11:18 am
michel– Actually, many people are very territorial, and that can apply to “their” data. They will reveal precisely as much as they are forced to reveal to get whatever benefit they value while witholding when they believe doing so might give them an edge to future benefits.
For researchers, the benefit is often recognition which can come in the form of publications, research grants etc. The motive for withholding isn’t necessarily fear that what they did will be shown wrong. The motive can be more subtle.
Here’s a hypothetical: Suppose someone other than Jones comes up with another surface temperature reconstructions. Let’s call it “SteveTemp”. As long as Jones does not reveal his methodology, no one can explain or interpret any differences between the two reconstructions. Since person who comes up with a new reconstruction is generally expected to compare and contrast his reconstruction to the pre–existing one, Jones secrecy makes it unnecessarily difficult for new people to get attention for a competing reconstruction. To some extent, this protects Jones standing in the field (after all, his is the proxy that is cited.) To some extent, this protects Jones funding stream.
In this hypothetical, none of Jones motivation is based on fear his proxy is bogus. But even this hypothetical might not be his motive. I can come up with many possible motivations, none of which involves Jones being even slightly worried his proxy will be shown to be bogus.
Andrew_KY (Comment#16736) July 26th, 2009 at 11:34 am
The Fraud Hypothesis is just as plausible as the Prima Donna Hypothesis. Furthermore, both could be true at the same time.
Andrew
lucia (Comment#16740) July 26th, 2009 at 11:42 am
Andrew_KY–
If your point is that we can’t exclude the possibility of fraud… well, no. But I generally tend to suspect other motives, as it’s my experience the other motives are more common.
There are many accusations of fraud hurled by both the stone-cold coolers and the hell-fire and brimstone warmers. The accusations tend to generate a lot of heat but very little light.
Andrew_KY (Comment#16742) July 26th, 2009 at 11:53 am
Lucia,
Your experience is anecdotal, I’m afraid. A much longer record of human history is full of examples of one person trying to get over on another or one group trying to manipulate another with lies. I don’t think people in your experience have any kind of special exemption from fraudulent behavior. You might like to think so, but wouldn’t we all for the people we know?
Andrew
lucia (Comment#16744) July 26th, 2009 at 1:16 pm
Andrew_KY.
Everyone’s experience is anecdotal.
History is also full of examples of people who are vain, foolish, selfish, bumbling, quest for glory etc. That fact that we also find evidence of lies and fraud doesn’t mean it’s more common than vanity, foolishness & etc.
Andrew_KY (Comment#16751) July 26th, 2009 at 5:16 pm
Lucia,
Indeed, but since we both know that those involved in fraud are not likely to admit it (especially while it’s ongoing), it remains just as much a possibility as any other explanation for odd behavior.
Andrew
michel (Comment#16773) July 27th, 2009 at 12:58 am
Lucia, I’d agree with you if it were just Jones, and if it were just this particular case from Jones. But it is not. The problem is its a pattern with Jones, and its a pattern with the whole RC team. I always used to think the conspiracy theories were totally hare-brained. But time has gone by, and there have been more and more episodes where these guys have refused to reveal, then finally been driven to revealing, and its turned out that the evidence is quite different from what was asserted.
We started this craziness with Mann, who, remember, has still not revealed his algorithm despite appearing before a Congressional Committee who demanded it. Why not? We go on to Thompson, a respected scientist and the inventor of ‘Dr Thompson’s thermometer’. Then there is the strange case of the bristle cone pines, and the Gaspe Cedars. We really do need to free the Gaspe Cedars, as well as the CRU data. There is Steig, Ramsdorf. There is the mystery of the Chinese surface stations, the FOI proceedings, and their meta-data which allegedly refuted UHI, but later turned out not to exist in a good half of the cases. There’s a lot more than this too.
Jones, if it were an isolated example, this particular case, could be a fit of pique, though an unusually prolonged and obstinate and self defeating one. The series of events however, smells strongly of fish. It makes no sense that they would really have all these piles of evidence that proves decisively what they want to convince us of, but all refuse to release their bits of it. Its the fact its common collective behavior of so many of them that prompts one to think the worst.
Well, we have to see. In the meantime, I think we should say that those who refuse to reveal both data and code should be totally banned from further participation in the IPCC work or policy studies. Only way to get to the bottom of this, one way or the other.
Raven (Comment#16777) July 27th, 2009 at 5:08 am
Michel,
As Lucia says, said this hiding of methods and data is most likely driven by a simple desire to maintain the status of the academic in the community. If the methods are kept secret any critic is forced to invent their own to replicate the results. If the critic comes to different conclusions the original author can always say the critic’s methods are “flawed” and their results are better. If they make their methods public and a critic finds an error then they have no recourse.
This approach works especially well when a critic is outside the climate science community because it allows them to use the ‘argument from authority’ to dismiss the critic’s methods.
This approach has been so effective that it has become an unofficial climate science policy (e.g. never give methods to sceptics) which creates the illusion of a conspiracy when it is really on result of individual decisions based on professional self interest.
lucia (Comment#16780) July 27th, 2009 at 6:59 am
michel…
Look, there are so many other possible explanations rather than a fraud, lying etc. Why is it everyone on all sides wants to insist that people they disagree with are propagating frauds, while the ones they agree with are motivated by something else? (Or just made mistakes… or whatever?)
I don’t think any of the guys you are accusing of fraud are intentionally committing fraud. I think some may have made mistakes; some may suffer confirmation bias. Some have sometimes been sloppy, failed to explain what they did when they should have (e.g. Rahmstorf failing to mention he changed M). I think sometimes they don’t know how their behavior looks.
But I don’t think they are committing fraud.
I’d like the data and methodology underlying HadCRut to be more transparent. I don’t feel any compulsion to delve the precise nature of Jones motives– it doesn’t matter on bit. Greater transparency is desirable regardless.
So, “Free the CRU data” is pretty much as far as I go on this one.
Andrew_KY (Comment#16782) July 27th, 2009 at 7:04 am
“creates the illusion of a conspiracy”
I’d say that any person or persons who know their behavior potentially creates the illusion of conspiracy is engaging in fraud.
Sounds ’scandalous’ to me.
1 a: discredit brought upon religion by unseemly conduct in a religious person b: conduct that causes or encourages a lapse of faith or of religious obedience in another
The purveyors of AGW need to get their house in order, don’t they?
Andrew
lucia (Comment#16784) July 27th, 2009 at 7:20 am
Andrew_KY:
Come on now. The definition of fraud does not include “behaving in a way that potentially creates the illusion of a conspiracy”. In particular, it does not mean “behaving in a way that potentially creates the illusion of a conspiracy in the minds of overly imaginative people who wish think all who disagree with them must be involved in conspiracies”.
You can’t redefine the word “fraud” to mean whatever you’d like it to mean so you can apply it to those with whom you disagree.
Scientists or politicians knowingly or unknowingly behaving in a way that potentially creates the illusion of they are involved in a conspiracy are often acting foolishly. But being foolish is not the sae as being fraudulent!
Andrew_KY (Comment#16786) July 27th, 2009 at 7:37 am
Lucia,
fraud (frôd)
noun
deceit; trickery; cheating
de·ceit (d-st)
n.
1. The act or practice of deceiving; deception.
2. A stratagem; a trick.
3. The quality of being deceitful; falseness.
Since when is having an “overly imaginative mind” a problem? Isn’t that what AGW is based on? An imaginative vision of the future?
Are you seriously saying that the odd behavior of some AGW promoters must be based on simple stupidity? That climate scientists are stupid? If not, their behavior certainly fits the definition of deceit, which is the first word in the def of fraud.
Andrew
lucia (Comment#16787) July 27th, 2009 at 7:49 am
Andrew_KY:
Note that your definition does not include ““behaving in a way that potentially creates the illusion of a conspiracy”.
Gosh… it’s bizarre. Here, I have Andrew insisiting I accuse Jones, Mann, Steig, R of fraud as the only possible explanation for anything and everything they do, and at Tamino’s I have others insisting I accuse Carter of fraud and deception for anything and everything he does. And oddly, over there they are complaining I accuse Mann, Steig and R of fraud! Sheesh.
Now, Andrew… First, you must be aware that odd behavior is not necessarily deceptive. Since when can’t stupidity be the explanation for odd behavior? Also, why are you asking if I am saying the behavior of AGW promoters must be based on stupidity? Stupidity wasn’t even in the list of qualities I included when I suggested possible motivations. Vanity… tribalisms… territoriality… are all possible motivations for odd behavior. I have no idea which motivates specifically motivates any individual person.
As I said before
Andrew_KY (Comment#16788) July 27th, 2009 at 8:01 am
Lucia,
I don’t insist you call anyone anything. I just think your explanations in these specific cases of human behavior are incomplete. That there is some deception (therefore, fraud) involved in promoting AGW is a significant possibility. It shouldn’t be dismissed.
Andrew
steven mosher (Comment#16858) July 29th, 2009 at 1:54 am
One reason why we publish data and methods is to REMOVE the issue of fishing about for motives. Motives are not observables. Now to be sure I’ve engaged in motive hunting. We all do when somebodies actions don’t make SENSE. If I release my data and my methods, then “I” and my motives can be isolated from the equation. You don’t like the way I ’sliced’ the data, make your own damn slices. You don’t like the way I did my QC? propose your own damn method. Are my results robust with respect to these choices? that’s a good statistical question. Is my method standard cookbook approach? tried and true? is my method limited? did I cook up some new statistical recipe? ok, does it hold water? what are it’s limitations.
When you reveal data and methods then the choices we make become the issue, not our motives for making them.
One thing I love about lucia’s work is that she work’s like I do, bad spelling and all. here are three ways of looking at this problem; A,B,C. here are the pros and cons, the assumptions, the limitations, etc.
lucia (Comment#16860) July 29th, 2009 at 5:23 am
Precisely. We don’t need to worry about motives. If the data and methods are clear to others to inspect or repeat, then people can figure out whether your results hold up. If everything is easily available, people can figure it out quickly.
I think one thing that gets lost in all of the climate-blog wars is that whether things can be examined quickly or slowly matters in the case of climate change. The speed with which doubters can verify your results is important, particularly if the doubters are expected to act on your results. Telling doubters “go do your own study” is not going to cause them to vote or support a policy based on study results they feel (for whatever reason) is flawed.
So, utter transparency if very useful in climate science currently.
Andrew_KY (Comment#16862) July 29th, 2009 at 6:50 am
“We don’t need to worry about motives”
We need to worry about motives.
When a crime has been committed and is investigated, investigators always examine what parties have motive + opportunity to commit the crime.
That idea doesn’t change when the crime in happening right in front of you.
Andrew
lucia (Comment#16863) July 29th, 2009 at 7:17 am
Andrew_KY–
We would need to worry about motives in most criminal trials. In those cases, there is often no crime unless a motive exists.
But, we don’t need to worry about a motive when figuring out questions like: Is the trend in the HadCrut data an artifact resulting form basing temperature measurements at airports? How robust is the temperature to inclusion or exclusion of certain sites?
Jones motive in keeping data to himself does not affect the answer to this type of question.
In my opinion, the questions of fact about issues surrounding the robustness of the trend are much more important than any issues about Jones motives. I think he’s acting like a weenie, but … well.. so? Maybe he’s a weenie. Maybe not. And why should anyone, including him, care if I think he’s behaving like a weenie?
What does matter is that his desire to exclude SteveM from seeing this data is polarizing, and amplifying suspicions. Giving access to the data voluntarily, without an act of Parliament, would solve so many problems. It would
a) remove a talking point from those with whom he disagrees,
b) permit Steve (and others) to see whether or not UHI matters, or see which stations Hadley really uses.
c) see whether inclusion or exclusion of a very small number of sites makes a large difference.
I am pretty darn sure the trend will still remain after SteveM (or anyone) looks at data. So, there is really very little downside to revealing the data. It should be revealed.
Andrew_KY (Comment#16864) July 29th, 2009 at 7:32 am
“But, we don’t need to worry about a motive when figuring out questions like: Is the trend in the HadCrut data an artifact resulting form basing temperature measurements at airports?”
If we were convinced The Trend is an objective conclusion, I agree. However, there is plenty of evidence that it is not.
Andrew