The Blackboard

My take Lucia.

The mails clearly show that jones and mann have serious issues surrounding their ability to judge things objectively and put aside personal interest. Even in something as innocuous as the “baseline” period, jones says he doesn’t want it changed until he retires. That’s funny, but very telling. ( just hunt for the mail in which they are discussing baseline periods… jones mentions 35 years and then says thats too hard to compute ?? very funny stuff) Anyways, Like I have always said I would reject any paper or any study that uses HADCRU temps. No data; no code; no credibility.
Now, that goes even further. If somebody points me to a paper I want to know this:

1. did phil jones or michael mann have ANYTHING WHATSOEVER
to do with this paper/analysis/data/code. producing it, reviewing it. anything whatsoever.
2. If you cant prove that they had nothing to do with the paper,
or code or data, then I would not touch it. false in one, false in all.

Finally, somebody needs to champion an appeal to the journals to change the way they do things

Do you thus reject UHA, Steven? I can’t find the raw data and the code…..

Simon Evans (Comment#24735) November 25th, 2009 at 3:46 pm

yup. I’ve pressed for that data and code as well. You’ll find me also over on WUWT pressuring Anthony to post code and data for the posts he puts up. Same here with Lucia. Sorry simon. It’s called a principle for a reason simon. You’ve obviously not seen me posting since 2007 on the issue of reproduceable results.

Hint, when steve McIntyre posts turnkey code, John A runs his code
and posts the figures that result. That’s called reproducing the results. Steve Posts a Graph. Graph A. he posts code that he says
generates graph A. Then John A does the first fundamental quality check: does the code posted run and produce the graph posted.

Now why do guys like John A and I think this is important? Dunno, In my case maybe cause I’ve written reports
several hundred pages long with hundreds of charts and I have some experience with the code and the figures not always matching. So. You want to convince me. Give me your data;
give me your code; lets see if you pass the most basic quality test.

I believe we need to go further than that. IPCC is thoroughly corrupted, so we need to establish a new organisation, which can do this. I have worked in the consulting engineers business many years, and think we need to put the job of assessing human impact on the state of the world to a consortium of big consulting engineering companies, which can look at the matter in hand with impartiality and detachment.

The funding and steering committee should come from diverse public and private sources, and must be relatively small compared to the economic strength of the consortium.

University research is not trained to do that, is often arrogant and narrowminded, and will always be too fond of their “Pet Theory”. Furthermore, universities can not construct professional databases, models and software in general.

In my country, Denmark, we do major EIA’s in that way, and very succesfully so.

I don’t disagree with you, Steven, that it would be a darned good thing for everone to post all their raw data and all their code in the first place. I am just pointing out, as you’ll realise, that this is not just a matter pertaining to whomever the ’sceptics’ (yes, I do use the word in coined quotation marks, since I am very unconvinced of the scepticism of so many of them) want to get their teeth into. I’m not even sure that Steve McIntyre has posted all his relevant code ever – are you?

Yet if you go to woodfortrees and plot the temperature for HADCRU vs the rest, the story is pretty much the same.

From this moment forward, any process that Mann and Jones participate in will be tainted in eyes of many.

Although this was already the case for many CA readers, the basis for this taint was wholly illegitimate in the eyes of the mainstream climate science community.

These emails have both spread and legitimized the taint. Before conducting any future assessment activities, the climate science community will have to ask the question:

If we are going to invest time in this process, shouldn’t we do everything in out power to make sure that the results of that time are as free from taint as possible?

Once asked, the only conclusion can be that participants in the CRU scandal must have their roles strictly limited. Otherwise, climate scientists are getting a poor ROI for their time.

Simon Evans (Comment#24742) November 25th, 2009 at 4:12 pm

WRT to Mac. I’ve never checked that he released all his code and data since I started reading him around 2007 or so. At some point during our evolving discussion steve started to post up code not only for his publsihed articles but blog posts as well. And more and more of his community has got in the habit of doing likewise.
nevertheless my principle still applies. If steve or anybody makes a post and doesnt or wont post the data and the code, then all I can do is suspend judgement on it. The nice thing about this principle is that personality is completely taken out of the picture.

Simon–
SteveM posts one heck of a lot of turnkey code. I suspect he’s posted all of it.

I’ve sometimes handed out excel spreadsheets for my posts. I’ll even hand them out if someone asks. But I don’t upload a new EXCEL spread sheet five times a month with the computations for just that post.

I’m in a middle ground on the “free the code” issue. I think codes should be made available under certain circumstances. CRU, GISSTemp, UAH, RSS codes should be if those results are going to be used in analyses in peer reviewed journals. But, what I think is more important is that if US tax payer money is used to create a code that creates a product like GISSTemp, UAH or RSS, then the funding agencies should ensure that the productions codes used every month should be publicly available and archived. The specifics required for snoopy curious nosy people to figure out precisely how the product is made should provided.

I don’t, on the other hand, think we need to post codes to say 2+2=4. Here’s the link to the code.

Jason (Comment#24746) November 25th, 2009 at 4:23 pm

yes jason. On the AGW side you will see people reject papers out of hand because they are published in E&E. Why? becuase of concerns about the editorial policies of E&E. And AGWers have argued that the science journals they publish in are beyond reproach. opps.

Lucia,

I know Steve posts loads of code – I was citing him as a ’shining example’, but pointing out that just maybe he doesn’t always post the last 2%, so to speak (I just say maybe – he might well do, for all I know).

I guess I’m arguing that !00% posting of data/code is not a sensible first gate to consider whether science is science or not. Shucks, did Darwin post all his data?

On the AGW side you will see people reject papers out of hand because they are published in E&E. Why? becuase of concerns about the editorial policies of E&E.

I don’t agree – I think it’s because of experience reading stuff published by E&E. It’s inclined to make one lack enthusiasm.

Simon–
I agree with you that 100% posting– particularly proactive is not necessary. I’ll give out codes under certain circumstances. But I also think that sometimes, it’s sufficient to say, “I got the data here. I averaged using the normal definition of “average”.

My readers have often repeated my computations on their own and replicated. I think quite often that is sufficient.

Other times it’s not. And in the case of CRU, I think they should have given out code. If this famous “Harry” code is the real thing, I think we now know why they didn’t!

Lucia,

I also think CRU should have given out whatever they could – I think any unnecessary obstruction has been hugely stupid stupid stupid!

If they were obstructing out of pique or whatever then they were extraordinarily politically naive.

I don’t happen to think they were hiding anything ‘needing’ hiding, but if the presumption of that springs from their stupidity then, d’oh, that’s why they were being naive…..

lucia (Comment#24750) November 25th, 2009 at 4:29 pm

HA. if you switched from Excell to R and posted your code then..

A. Other people could take your code and enhance it ( make sure to use a copyleft license)
B. Other people could beat you to the punch and post your analysis before you ( incentive for you to be fast or give the job to somebody else)
C. Many eyes to error check your butt.

D. Some smart bunny could automate the whole damn process for you, so that when hadcru updated their numbers your little code would launch and presto.. new charts and graphs. of course you check before publishing or not. your community of eyes could do the checking for you.

Kinda kool we could make a lucia bot that just watched for changes in hadcru and spammed me with lucia plots.

Its all about taking the humans out of the loop. Misanthropic I am. or lazy take your pick

bugs:

Yet if you go to woodfortrees and plot the temperature for HADCRU vs the rest, the story is pretty much the same.

HadCRUT is the tip of the iceberg. It would be nice to have the raw data and code publicly released, because that means it can be improved by other people besides Phil Jones.

The real thing would be for people Mann, Steig, Santer et al., release turnkey versions of their code.

Really if you don’t have turnkey ability, I don’t know how you know when you’ve got it correct. My stuff is always done that way on anything I take seriously enough to publish at least. Even for some of the graphs I post on this blog, I have the shell scripts preserved that go with them.

The beauty of scripts and full data set (including any metadata needed to reproduce the results) is they fully document everything you’ve done, from whether you skipped part of the file when you analyzed it, to whether you used artifact rejection, data window length and so forth.

And it allows your colleagues to not only reproduce what you’ve done by to see the effects of tweaking assumptions that you’ve made.

“yup. I’ve pressed for that data and code as well.”

I’ve never seen you press for UAH’s code. Maybe I missed it. I doubt you press for it with as much zeal. Or do you? Linkies would be helpful.

Carrick,

The real thing would be for people Mann, Steig, Santer et al., release turnkey versions of their code.

But not Spencer and Christy? Are you simply a hypocrite?

Steve– Switching to R doesn’t change the principle. I don’t think every single thing needs to be documented by providing codes. I think some codes need to be provided and others don’t.

Boris–If you want it, why don’t you ask? There is no point complaining you can’t get something if you don’t even ask for it.

Carrick– I think once Santer was forced to provide his temperature of the tropical troposphere, that was sufficient for any analyses I wanted to do. Would it have been even better if he’d provide codes? Sure. But at a certain point, making people provide every conceivable thing will become overwhelming.

Boris (Comment#24760) November 25th, 2009 at 5:00 pm

Nope not with much zeal at all. Maybe once or twice prolly on WUWT or sending msgs through Anthony. For example, when Menne came out with his new approach for homogenization at NOAA and I knew that Anthony was personally meeting with the scientist I asked Anthony to ask him in person. Also asked him to ask Tom Karl for code as I recall. Usually my requests start with a personal appeal to the scientist in question. If I know somebody who knows them personally I’ll put the request in that way. I’m most vocal about the temperature record ( land record) because.

1. I understand it better.
2. It’s core to almost everything.
3. It’s a simple way to demonstrate how open science can work.
4. Personal private appeals to the scientists in question failed.

So in principle you wont find me making exceptions for IPR or confidential climate data or bad people who misuse data. In practice I have to ficus on the areas that I think best achieve my goals.

Simon:

But not Spencer and Christy? Are you simply a hypocrite?

Jeebus, take a valium.

Which part of et al. are you having trouble with today?

Of course Spencer and Christy.

It is interesting that Mr. Mann is using Von Storch’s resignation from his journal as supporting the peer review process as Mann/Jones/Schmidt define it, while Dr. Von Storch is clearly concerned that they degraded the process themselves. And, Simon, I appreciate that you strive to hold the skeptics and alarmists to the same standards, but you don’t need to be so snarky about it. It makes you sound like Gavin or Grant.

lucia (Comment#24764) November 25th, 2009 at 5:21 pm

The problem with just requiring some codes and data is illustrated in steve McIntyres request to get intermediate values from certain computations. Another example is the TOBS correction in the data that GISS uses. NOAA takes “raw” data and applies a TOBS adjustment. This data then is imported by GISS. GISS call it RAW. when in fact its processed. So, I’ve asked Tom Karl for his code written long ago to do the TOBS adjustment. No Joy.

I can probably link to some guidelines for reproduceable results.
But the principle is that the results need to be reproduceable.

Lucia:

Carrick– I think once Santer was forced to provide his temperature of the tropical troposphere, that was sufficient for any analyses I wanted to do. Would it have been even better if he’d provide codes? Sure. But at a certain point, making people provide every conceivable thing will become overwhelming.

if you approach it turn-key from the start it isn’t that bad. I’m not saying to provide everything, just the codes to reproduce published results.

As you know I deal with pretty large data sets with lots of files and metadata myself. There are enough collaborators that turn-key is a requirement.

I can’t imagine keeping that all straight using just a GUI interface.

Every publication these clowns have produced is now suspect. Also publications by others that reference clown publications for some sort of support.

Boris:

I’ve never seen you press for UAH’s code. Maybe I missed it. I doubt you press for it with as much zeal. Or do you? Linkies would be helpful.

LMAO at “with as much zeal”.

Is this science or creative dance?

Come on Lucia show us some emotion and follow-through with that 3-move-combo FOI request!

Steve–
If they’d given the stuff over, then your example problem goes away.

Also, GISSTemp is specifically a product whose code falls in the category that I think should be publicly available. That includes everything required to go from station data to gridded and global surface temperature values. The reason that need to available is:
1) It is tax payer funded.
2) It must be a production code since it’s used to create monthly values.
3) The GISSTemp product is an important producted used by other researchers downstream.

But you are trying to use the exmaple of GISSTemp and intrasigence of some public employees to craft an overly broad requirement that says if someone posts ” the sum of intergers from 1 to 10 inclusive is 55; I computed this using EXCEL” then that person has to post the EXCEL sheet. Or worse, they have to provide a turnkey code in R. This is nuts. There has to be some intermediate criterion other than “free the code”.

Joel (Comment#24768)

I’m getting there, Joel. It’s something to do with the taste in my mouth.

Lucia:

But you are trying to use the exmaple of GISSTemp and intrasigence of some public employees to craft an overly broad requirement that says if someone posts ” the sum of intergers from 1 to 10 inclusive is 55; I computed this using EXCEL” then that person has to post the EXCEL sheet. Or worse, they have to provide a turnkey code in R. This is nuts. There has to be some intermediate criterion other than “free the code”.

There isn’t a better one unfortunately, because the alternative, “provide enough details of the algorithm to allow reproduction” is not verifiable.

I think you are making too much of the difficulties in providing turn-key code. As far as I understand, that’s even possible with Excel scripting.

lucia (Comment#24756) November 25th, 2009 at 4:46 pm
” But I also think that sometimes, it’s sufficient to say, “I got the data here. I averaged using the normal definition of “average”.”

A couple of points. I argue for the data AS USED and code AS USED because pointing at the data is not enough. You should know this. Every month when hadcru updates you are aware that the past data changes. WRT verbal descriptions of algorithms people tried this on me for a long time. the text did not and could not describe the algorithm. and we could never agree to this. Typically the code is required because text is ambiguous.

carrick you need more than the code to reproduce the published results.

how about all those little trial and error regressions you did to data snoop.

“Von Storch also said that the University of East Anglia (UEA) had “violated a fundamental principle of science” by refusing to share data with other researchers. “They play science as a power game,” he said.”

They are caught up in a power game, he means.

http://en.wikipedia.org/wiki/C.....g_incident

McIntyre and Co have succeeded quite well in dragging science down into the mud. Climate Audit is a continual stream of snide references to individuals, and egging on the cheer squad. He uses the well known and ancient rhetorical device of “I’m not saying it’s ‘x’,” where ‘x’ is what he is accusing them of. The audience certainly responds to the device perfectly.

Steven Mosher:

carrick you need more than the code to reproduce the published results.
how about all those little trial and error regressions you did to data snoop.

Technically, no.

You don’t need that to reproduce the results. You may need that to understand their parameter choices.

But again, if I give you the turnkey scripts, you can modify them to try other parameter choices. In my experience this works well enough.

Carrick–
Very large amounts of data need turnkey code. But some papers might be based on smaller amounts of data and don’t need a turnkey code. There is no point in making rule overly broad.

Steven–
I understand that sometimes the verbal description is not enough. However, it sometimes is. The verbal description of how to apply the red-noise corrected t-test to monthly data in Santer was enough to produce the results. In these cases, I don’t think a turnkey code should be a requirement based on the principle that sometimes people don’t describe what they do.

I have all sorts of issues with climate papers that don’t describe what they do well enough. This even occurs when the authors have done something fairly simple. But my preference would be for them to stop writing these ridiculously short papers and start actually describing what they do if at all possible.

I don’t think people should be communicating certain things in Fortran, R, C, Matlab etc. Some should be communicated with words and equation. Focusing on “free the code” can have the effect of making the verbal discussions worse, not better.

bugs:

McIntyre and Co have succeeded quite well in dragging science down into the mud.

Swish.

These emails go back to 1999, so the tactics precede McIntyre’s entry into the field.

Lucia:

Very large amounts of data need turnkey code. But some papers might be based on smaller amounts of data and don’t need a turnkey code. There is no point in making rule overly broad.

If you can demonstrate that the algorithm is simple enough and described in sufficient detail to allow it’s replication, I agree. That should be an assertion made by the authors and verified by the referees and editors.

But it does open up the process to abuse.

Carrick–
I’m not saying it’s too difficult. I’m saying sometimes the requirement is both unnecessary and becomes ridiculous.

I think you are making too much of the difficulties in providing turn-key code. As far as I understand, that’s even possible with Excel scripting.

Of course it’s possible. But I still think Steve’s notion that I must upload, archive and provide a link to a specific spread sheet for every blog post is unnecessary. If he want’s to suspect judgment, that’s fine with me. A larger number of people have just downloaded and reproduced– asking questions when they wish too.

The only real impediment to my providing the spread sheet is this. Because it’s EXCEL, I keep a large number of analyses running in parallel. But even though EXCEL has some advantages, it has disadvantages. One is that my auto-plot with hadly/GISS/NOAA all in it “breaks” when I get the GISS data but HadCRU and NOAA aren’t yet available. So, I duplicate that worksheet, delete GISS and NOAA and have a good graph. But, I often have a number of NOAA and GISS computations that are currently wrong because the months data is wrong.

If I post these, (as I have in the past) there are people who always ask about the sheets I did not use. So… I would have to go and delete all the parts that don’t have to do with the post etc. I would do this for a journal article but not a blog post. Sorry, no.

If someone specifically wants an answer to a question I’ll answer it. And if they want a specific spread sheet (before next month comes along) I will do all the appropriate deletions to give them one that is not confusing. But otherwise, I am not spend more time prettifying the spreadsheets than updating to post the interesting information. Just… not… going…. to … do it … for … a … blog…post. Nope.

This is entirely ludicrous. If you were to look at any other field of science you will find a lack of readily accessible raw data and code. The ‘requirements’ that are being proposed are specifically to do with the matter of pressure upon this particular area of science. There is a case to be made for that, for sure, because the implications are so significant, but proposing the notion that this has not been proper science in the first place is ridiculous, unless you take the view that there has never ever been any proper science.

Which field do you want to consider? Evolution is evidently a load of baloney – where’s the data, where’s the code?

By all means press for access in this area – whilst at least having the decency to recognise that that pressure is being responded to – but suggesting that there has been a particular issue in this field is a product of negative wishful thinking. The GISS data has been made public – what was the outcome of that? Big nothing. It seems to me like a desperate pretence that something smoking-gun-like is being concealed. Discount HadCRUT if you wish to – who cares? Or if they get around to sorting out agreement on releasing the final 2%, and then there turns out to be nothing of consequence having been ‘concealed’, what then? What will be the next thing that you wish to be revealed in the pretence of suggesting that there is some deceit afoot?

Of course standards can be better, and of course they should be. But this is being driven by an ongoing pretence of pushing for better standards which is in truth a generation of doubt being the product.

Do you not think it might be better for some people to come up with some science?

Nevermind on this…been covered. I should refresh my browser more than once a day

“I’m not even sure that Steve McIntyre has posted all his relevant code ever” – Simon, my experience is that McIntyre does posts his scripts (typically in the first comment under the general post). Perhaps this hasn’t occurred on ‘every single’ post, but it is his MO, as far as I’m aware. Are you aware of a single instance when someone has questioned SM on his approach and/or code and he has appeared to hide anything?

@Simon Evans (Comment#24789)
“This is entirely ludicrous. If you were to look at any other field of science you will find a lack of readily accessible raw data and code.”

That’s not my understanding of life critical systems such as aeroplane control and technology or drug trials. Isn’t the proposition AGW threatens global devastation and loss of life on an unprecedented scale?

Steven Geiger

Steven,

I’ve said above that I was citing SM as a good example but wondering whether he’s always posted every bit of his code.

I don’t know whether he’s always posted everything, perhaps he has. It simply seems ridiculous to me to demand that everyone should always post every bit of code they ever use. Call me naive, if you will, but I”ve been inclined to trust the idea that Einstein could handle basic arithmetic.

“Do you not think it might be better for some people to come up with some science?”

Simon, this might be good enough for you. But for a lot of us here in the material world, where things have to actually work, it’s not good enough. To make an assertion and call it “science” isn’t good enough.

Make an assertion, then demonstrate that it’s true, or allow people to show that it’s not true.

If it can’t be demonstrated either way, then it’s just an opinion, and no one should be bound by it or pressured into believing it.

Andrew

curious (Comment#24792)

That’s not my understanding of life critical systems such as aeroplane control and technology or drug trials.

Ok, curious, give me links, please, to, for example, all the drug trials raw data relating to fluoxetine plus all code associated with the analysis of that data.

If you’re not making a bullshit statement then I look forward to your links to sources (and no, I don’t mean to papers, I mean to raw data and code, thank you).

Simon
Based on the analysis already done on the CRU program it’s probable that Phil Jones is incapable of even reproducing his own results much less capable of handing data and code over to someone to check the raw data and gyrations he put the data through. That does not inspire a lot of confidence in the temperature record much less studies that are based on it.

For everyone’s information James Hansen in an interview has already said that the data and code needs to become transparent. If the data is proprietary to the CRU and they are unwilling to release it that’s fine. However, studies that wish to provide the basis to radically modify the world economy will have to be based on a database that is publicly available and reprodicible. If the IPCC is the basis for world changes CRU data products will just have to be excluded for studies included in their summaries.
Sound fair?
THanks
Ed

Ed,

I think that’s fine. I would be very happy never to hear of CRU ever again. Ok? They don’t make a decent fist of covering the Arctic anyway.

Von Storch validates the hockey-stick

http://www.sciencemag.org/cgi/.....6/5696/679

and confirms climate change is a serious issue.

We are facing an interesting phase in the postnormal science of climate, where the ideological debate between the camps, the self-serving wagon-circling of others, and the understanding that climate change is a serious issue and needs our attention not only in general “protecting climate” (i.e., reduction of emissions) but regional and local adapatation as well.

http://coast.gkss.de/staff/sto.....r.2003.htm

The problem appears to be one of being too sensitive to denialists, and wanting to take them head on. I met a climate researcher once and asked him about his attitude towards denialists. He just laughed and said he completely ignored them. He won’t ever get caught up in the pig sty, but at the same time, the deniers are swaying important political figures and public opinion to their non scientific fantasies.

Von Storch also resigned from the Journal that is mentioned in the emails that they thought should not be published.

http://coast.gkss.de/staff/sto.....r.2003.htm

Simon – the FDA should have that:

http://www.fda.gov/Drugs/Devel.....efault.htm

Lucia, were you answering me or having a swear-moment at Excel?

Just kidding. You’ve outlined why I don’t use it.

When I said “published article”, I meant journal articles not blog entries. You are free to add enough details, and we can ask questions in context about what you’ve done, that usually i can figure out what you are doing.

Anyway, for myself, turnkey is part of my verification process. And anytime I hand over data archives (they are big enough they usually go as a removable hard drive), it includes the scripts I use.

For example (modified from a real-world example):

Analysis
Bin
Calibration
Data
Documentation
Metadata
Photos
Readme.txt
Results
Talks
Temp

These are all directories. (Temp=temporary files used in building other results. These are kept as part of the build process but can be safely delete.)

bugs:

I met a climate researcher once

Must have been your big day. XD

Carrick

That should be an assertion made by the authors and verified by the referees and editors.

But it does open up the process to abuse.

All process are open to abuse. Even the notion that “all” codes and “all” data would be required is open to abuse. You know perfectly well peer reviewers aren’t each going to run a script, download gridded data, run it for days to replicate etc. So, researchers could easily “make mistakes”… ooopss! Or claim that they put the right thing up, and that the person unable to re-run is just not competent. Or something.

You can’t get to perfect.

My position is that we don’t need everything posted for ever freakin’ blog post. Steve Mosher is trying to elevate this to a ridiculous standard that can actually interfere with brainstorming ideas. While one might suggest that blogs will never have the respectability of archival journals if they don’t meet his documentations standards, the fact is that for the most part, they won’t anyway for many, many reasons.

Lucia:

You know perfectly well peer reviewers aren’t each going to run a script, download gridded data, run it for days to replicate

I’ve done that a few times (using my own scripts), but it was because the editor asked me for input on that.

curious (Comment#24802)

No, the FDA doesn’t have that, and that’s precisely my point. No raw data, no code relating to its processing. The FDA simply gives you the results of clinical trials. Be honest, curious, that’s all that’s there.

But I agree with this “My position is that we don’t need everything posted for ever freakin’ blog post”". I don’t think you even need it for blog posts, since the author normally has a cordial relationship with her audience (or most of them anyway).

The problem is when you get into adversarial relationships, disclosure gets complicated. It has nothing whatsoever to do with McIntyre or anybody else. That just happens when the stakes get high.

bugs–
Huh? The paper you link says:

The centennial variability of the NH temperature is underestimated by the regression-based methods applied here, suggesting that past variations may have been at least a factor of 2 larger than indicated by empirical reconstructions.

This is the opposite of Von Storch validating the hockey stick. He found that the straight shaft was not to be believed. The method underestimated the wiggles.

Even the slickest car salesman at least has a product that he can demonstrate to you, and that you can experience for yourself at some level.

What do Global Warming Hoaxsters have? Hockeystick drawings that indicate the world is going to end? Please… I have more respect for the car salesmen.

Andrew

Carrick–

Just kidding. You’ve outlined why I don’t use it

I use it for somethings and not others. If a problem gets too big, it’s just something to swear at. But R is butt slow, and has it’s aggravations. So, we can code fortran etc.

For what I do each week when UAH, RSS etc come out, EXCEL is actually pretty fast. It’s not even necessarily a bad thing for documenting since all I’m looking at is monthly data and the data get slapped into individual work sheets. So, everything is stored in one place.

But.. I post at what might be called ‘intermediate’ times in terms of the life of the EXCEL sheet. So, there are “issues” with uploading for every freakin’ post.

When I said “published article”, I meant journal articles not blog entries.

Yes. But Steve Mosher is saying that this sort of thing should be provided blog posts even for trivially simple computations. That’s where I’m saying his standard is overly broad and too time consuming.

Simon – Are you saying the FDA do nothing beyond look at output results provided to them? I thought they checked trial design and methodology, trial results and conclusions drawn? Are you saying they recieve a piece of paper from an applicant which says “this works and it is safe – we’ve tested it” and go no further?

Perhaps we are talking at cross purposes but I read your comment with the bolded “any” to be an assertion that auditable systems and results are the exception. My response was based on two areas where my understanding is they are the norm.

“Steve Mosher is trying to elevate this to a ridiculous standard that can actually interfere with brainstorming ideas. ”

I will agree with that to some extent but there is also another way of looking at it. You can say “here is the code I used to create that result” and yeah, maybe it won’t run because of some cruft that is particular to your environment (looking at some remote mount for the directory containing the data files, etc) and those sorts of things can be expected to be cleaned up by the person running the code in their environment.

In fact, a lot of “open source” code development happens just like that. Someone posts something that is absolute crap, people start chiming in with patches to clean it up and before you know it, you have something that is much cleaner. Maybe now the program ASKS you where the data files are instead of assuming they are on the M: drive.

The point being that the requester should be prepared to hack out the various housekeeping details to get it to run but the core routines that do the math/manipulation/adjustments/whatever should be the key point of the exercise and those should work as expected.

Having something perfectly “turn key” is probably asking too much. But people should also not be afraid to release warty code that doesn’t look pretty, either. A climate researcher isn’t a CS major and shouldn’t be expected to code at the same standard that a CS major would. But opening the code up might give that researcher access to some CS majors who wouldn’t mind contributing some spare time to help clean it up considering how important the field of research is these days.

And it doesn’t even have to be done to the general public. A university could put the code up internally and offer undergrads an opportunity to hack it into shape for extra credit. Chances are they will spot errors and fix them … like routines that error and don’t report the error to the caller or code that is never reached, etc.

It doesn’t have to be perfect but I would rather have 60% of something than 100% of nothing.

Steve,
Freeing the code is the first step but the bigger problem is the complete unwillingness amoung the climate science community to accept that non-climate scientists looking at the code can have useful things to say.

I use GISS as an example. It is pretty clear from the discussions I have seen at CA that the UHI adjustment algorithm is an ad hoc POS that manages to eliminate UHI discontinuties by smearing it across all stations – good and bad. Yet I have seen no evidence that these flaws have been acknowledged and I suspect that it would be ignored like the Micheals and McKitrick paper if someone bothered to write it up.

IOW – we need more than simply opening the code. We need accountability. Hansen should not be the one who decides whether a criticism of GISS Temp is valid or not.

Simon
In defense of Steve McIntyre, he asks researchers for intermediate steps from their research in those instances where they are using statistical forms of analysis that seem inapropriate to the application and the data. In many instances these researchers have “bastardiszed” good statistical procedure or used inappropriate procedures to tease results when the data just will not give them the warming result they desire. The Wegman report comes to mind. Designing a process that will produce a hockey stick result no matter what data is fed into it does not seem like a robust result I would want to base modifying the world’s economy on.

Bugs
I understand that climatologists are all laughing it up as we send them billions of tax dollars to keep them employed while they recommend putting my employer out of business. I’d like a little better accounting for how my money is being spent in secret so I’m not very sympathetic to a researcher whining about having to post everything necessary to completely replicate his results. If he doesn’t like it he can get a job in industry where he’ll have to do it anyway just to keep his job or get a contract or prove conformance.
Thanks
Ed

Nick Stokes

You release code, then have to explain every comment that mentions “adjustment”. And explain again. And so on.

Well..but you have to admit if that Harry code discussed at Anthony’s blog is real and not faked, it isn’t a matter of just calm note saying “adjustment”.

(I’m inclined to believe that file was juiced. I mean… OMG!!!!)

curious (Comment#24818)

Curious,

I’m simply pointing out that we, in the ‘blogosphere’, don’t have access to the raw data in clinical trials. I was quite interested in the effects of fluoxetine a few years ago and really could get nowhere further than an account of results.

I don’t know how deeply the FDA look into the raw data and the methodological process. I guess I have to trust them to do a good job on that! I think that is, generally, something we have to accept if we are being realistic about our lives. I’m pointing out, simply, that we do offer that acceptance in most fields, and so I think it is deeply unreasonable to jump to presumtions of potential deceit when we don’t have ready access to raw data and code in this field we are discussing. I think that ‘climate science’ is making considerable efforts, generally, to make its data and code freely accessible, and that in comparison to other fields of science. It’s not there to everyone’s satisfaction, but it’s much closer to being there than in other fields, IMV.

Crosspatch
I disagree with you. These climate researchers are being amply funded by tax dollars and work at a university where they have access to professors expert in computer science and statistics. There are plenty of CS grad students that can clean their code to a minimal open source standard. Their statistical methods should also be reviewed by a stats prof for “robustness” as recommended by Wegman. The world should not have to rely on Steve McIntyre to do all the work for them.

This actually gets to the crux of the problem, these researchers cannot share because the know how weak the studies and methods are. By maintaining a tight knit insular community that peer reviews amongst themselves they have a self reinforcing group impervious to scrutiny. Their argument against skeptics is that there is no published work that contradicts them. That’s no surprise since they will not allow it to be published and they’ve run people like Storch out of the field among others.

I believe these emails are just the tip of the iceberg and thank god they have been divulged even this late in the game.
Thanks
Ed

Simon–
I think the FDA has pretty strong guidelines for archiving, storing etc. of data. I’m not sure “we in the blogosphere” don’t have access. It may well be that if we asked someone, we could get it. There are some privacy considerations (since these are medical records involving people.)

The test of openness and transparency is: If people ask, can they get the data they feel they need? If they can’t, why not? And are the stated reasons why the data are excluded reasonable. (For example: it makes sense that some personal data might be kept highly confidential because it involve individual patient histories. But it would not make much sense if the PI-doctor involved in trials believed he could hand out anything including individual patient names, ages, home addresses,and medical histories with impunity to his buddy-doctors, but excluded other people because he claimed other people were not sufficiently meritorious to get the data. And it would not make sense if the PI-Doctor could just keep the parts that should be public and treat it as his private property.)

Why should non-experts get involved? The following is from a document called hot-proposal.doc, in the UEA documents.
.
Some restraints on participation
.
In contrast to the claims in favour of participation, some more reserved notions on stakeholder participation have been made.
.
Incapability of a rational judgement
.
A mainstream argument in political theories of the early 20th century (especially Schumpeter’s view on democracy) is that the ‘average’ citizen is not capable of a rational judgement on com-plex matters that go beyond the experiences of his daily life. Especially in matters which involve norms and values such as politics and many environmental problems, Schumpeter considers the policy preferences of citizens merely manipulable opinions which change with the issues of the day. At best, citizens will only be – to a limited extent – rational, if their own self-interests are at stake (Schumpeter, 1942; referred to in Hisschemöller, 1993).
.
Anyone else get a chill down the back of your neck? Anyone else feel insulted?

Simon
Here is a link to the FDA research data standards
http://www.fda.gov/downloads/F.....189445.pdf

The second page goes on to explain the data has to be in a SAS open format and that all data raw and derived must be included.

I’d say it would be a good idea for the EPA, NOAA, NASA and others that fund all climate reseach in the US from my tax dollars to implement a standard transparent format.
Thanks
Ed

Curious and Simon,
I spent a large part of my career working for the FAA in the certification of new airplanes. I believe the situation there is similar to the drug testing and approval process in that the company applying for approval of either airplane or drug has intellectual property rights and concerns. While all data and methods must be described and demonstrated to the authorities, the data remains the property of the company. Access to the data by the authorities is not restricted, but release of that information to the public is very strictly restricted, even to protection from FOIA release. So, if my reasoning based on experience is right, you are both right to a certain extent. In the case of the CRU data, keep in mind that the taxpayers are paying for the research and therefore own (or should own) rights to the data.
Disclaimer: Please remember that I am reasoning by analogy to my experience in a specific field and my experience may not apply in the world of drug testing.

On a related note, in the same document, its shows participants from Tyndall, Tata and RIVM (and others) as being paid 400 to 600 euros per day.

There should be some positions opening up soon. I must pop my CV in…

Ed,

Fair enough, I’ll cede that point. I agree with your principle.

I’d like to see peer review become open and transparent by emulating something like what’s happening at CA with data and code made available along with analysis and discussion. Sort of an online journal that’s iterative. Yes, it will be messy and have issues of credit and priority, but I doubt these are insurmountable obstacles. The obvious advantage is that more brains get to work on the problem. Unless the process is totally open, however, the lingering doubts about gatekeepers and political influence will keep it suspect. Way upthread Steven Mosher asked for somebody to appeal to the journals to change the way they do things. How about this idea as the target?

I’m cross-posting this from my blog because I haven’t seen it anywhere else yet. Please see my blog for a better formatted version. http://magicjava.blogspot.com/.....genda.html

It discusses Shell Oil and the WTO driving the climate research agenda. Apologies if this information is already known.

Source: uea-tyndall-shell-memo.doc

Mick Kelly and Aeree Kim (CRU, ENV) met with Robert Kleiburg (Shell International’s climate change team) on July 4th primarily to discuss access to Shell information as part of Aeree’s PhD study (our initiative) and broader collaboration through postgrad. student project placements (their initiative), but Robert was also interested in plans for the Tyndall Centre (TC). What ensued was necessarily a rather speculative discussion with the following points emerging.

Shell International would give serious consideration to what I referred to in the meeting as a ‘strategic partnership’ with the TC, broadly equivalent to a ‘flagship alliance’ in the TC proposal. A strategic partnership would involve not only the provision of funding but some (limited but genuine) role in setting the research agenda etc.

Shell’s interest is not in basic science. Any work they support must have a clear and immediate relevance to ‘real-world’ activities. They are particularly interested in emissions trading and CDM.[Clean Development Mechanism]

From: “paul horsman”
To: m.kelly@uea.ac.uk (Mike Kelly)
Source: greenpeace.txt

It was good to see you again yesterday – if briefly. One particular
thing you said – and we agreed – was about the IPCC reports and
the broader climate negotiations were working to the globalisation
agenda driven by organisations like the WTO. So my first question
is do you have anything written or published, or know of anything
particularly on this subject, which talks about this in more detail?

For a start the FDA standards from the above link is specific on how to name and label datasets, how top organize them, how to uniquely indentify and define data.

I see no reason why studies that wish to get published on climate could not also meet high standards like these. Researchers might have to cut back spending the grant money on flights to AGW conferences and spend it instead on paying a CS grad student to spend a few hours to keep their data compliant.

Of course papers coming from CRU and Michael Mann should be subject to intense scrutiny given that they are the basis for expensive policies that the public are being forced to face. If these papers are not peddled by the IPCC to the public, who’d care about the replicability of their results?

”
Anyone else get a chill down the back of your neck? Anyone else feel insulted?”

I can understand why you would and I don’t.

Bugs: you don’t feel insulted? If you don’t, then you should immediately remove yourself from the discussion, as you seem to agree with the statement:
.
the ‘average’ citizen is not capable of a rational judgement on com-plex matters.

Looks like the NZ AGW have been found cooking the books as well
http://briefingroom.typepad.co.....aking.html

“This is the opposite of Von Storch validating the hockey stick. He found that the straight shaft was not to be believed. The method underestimated the wiggles.”

You must have missed the first line.

“Empirical reconstructions of the Northern Hemisphere (NH) temperature in the past millennium based on multiproxy records depict small-amplitude variations followed by a clear warming trend in the past two centuries.”

bugs–
Uhhmm… No. I didn’t miss that.

Have you ever played hockey? Seen it played? We played field hockey in highschool. For that reason, I am aware that a hockey stick is not defined by the blade alone. You need the straight shaft too.

No one needs paleo to know the temperatures rose recently. The blade not the important part of “the hockey stick”. The important part is the shaft.

So, the finding that the shaft was wiggly repudiates ‘the hockey stick’.

It really doesn’t matter if someone provides “turn key R code”, if they provide an Excel spreadsheet, or just crayon scribbles on the back of a piece of paper. The real issues are if 1) the method of calculation is transparent and 2) if all the data needed to duplicate the calculation is provided.

As far as I can tell, many credible individuals (including Steve McIntyre, Jeff Id, and our gracious hostess Lucia) are pretty good about this sort of thing, while certain others, including Mann, Briffa, Santer, Jones, Gavin, ad nausea, et al, carefully and consistently hide the details of their calculations, specifically to make it difficult or impossible for a third party to duplicate/check their results.

For goodness sakes, nobody can pass a test in high school algebra without showing their work! There is no reason we should hold climate scientists to any lower standard.

Since all these folks are all publicly funded, there is a simple solution. Prohibit by law public funding of any research which does not provide immediate, complete, and transparent disclosure of a) raw data, b) calculation methods, and c) software code (whenever used). We taxpayers who fund the research deserve nothing less.

lucia (Comment#24825)
Well..but you have to admit if that Harry code discussed at Anthony’s blog is real and not faked, it isn’t a matter of just calm note saying “adjustment”.
No I don’t. As often, you’re reacting without reading properly. Here’s what it said:
Uses “corrected” MXD – but shouldn’t usually plot past 1960 because these will be artificially adjusted to look closer to the real temperatures.
Now there are two parts. “these will be artificially adjusted to look closer to the real temperatures” sounds ominous but is muddled – it’s a science issue, and will not be carefully explained in comments, especially by someone writing a graphics program. But it’s immediately countermanded by the first part “shouldn’t usually plot past 1960″. That makes no sense if you think someone is trying to rig the plot. What’s the point if you don’t show it?

In fact, this is related to previous discussions that we’ve had. What do you do smoothing near the end? Many say, stop when your symmetric filter runs out of points. Others try to use padded values to take the smooth further (or something equivalent).

This is actually a bit different, because it relates to divergence, which was much discussed at the time. But I think the smoothing issue, using instrumental values for padding, is mixed in as well.

WUWT now has a post with some other versions of this comment, and one refers to Osborn 2004. I presume they meant the Briffa/Osborn review paper. This sets it out in some detail:

The above facts seem to support an inference that some slowly varying factor began to exert a very widespread negative influence on the trend of these MXD data from around the middle of the 20th
century, with effects at higher frequency also becoming noticeable in some high-latitude regions. For the
time being, we circumvent this problem by restricting the calibration of the density data to the period before
1960. This reduces the potential overlap between temperature observations and density measurements and means that less data can be reserved for independent tests of the validity of predictive equations. This situation is far from ideal, but the alternative, using data after 1960 and thus incorporating non-temperature-
related bias when fitting regression equations as a function of density variability, would invariability produce earlier estimates of past temperature that, to some extent, too warm.

That’s actually talking about calibration rather than plotting, and divergence rather than smoothing, but it seems the same considerations apply. And that’s what the comment is saying. Adjustments have been tried, but are not ready to be shown.

Nick–
Why are you picking just one comment?! Come on. Sheesh.

Lucia,

“If I post these, (as I have in the past) there are people who always ask about the sheets I did not use. So… I would have to go and delete all the parts that don’t have to do with the post etc. I would do this for a journal article but not a blog post. Sorry, no.”

Get with the modern virtual machine world and run half a dozen virtuals with separate excel runs!!!!

Just make sure you have enough memory and licenses for everything!! ;>)

Nick Stokes,

“In fact, this is related to previous discussions that we’ve had. What do you do smoothing near the end? Many say, stop when your symmetric filter runs out of points. Others try to use padded values to take the smooth further (or something equivalent). ”

so, you are saying that between 1960 and say 1999 we are talking about an end point smoothing issue?? 39 years?????

HAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHAHA

Oh, and Nick, saying this is a calibration issue is ignoring the issue of whether trees of ANY type have been validated as thermometers over ANY period!!!

Keep slinging that hash old boy!!

“LMAO at “with as much zeal”.

Is this science or creative dance?

Come on Lucia show us some emotion and follow-through with that 3-move-combo FOI request!”

mosher is certainly a zealot in terms of releasing the code for the surface record. NO ONE is asking for UAH code. No one. Because you guys like it. It comforts you in your darkest moments, as the hordes of socialist bureaucrats ignite their torches on the horizon, you look to it for solace that things aren’t really what they seem.

Where’s the actual skeptic who was skeptical of the UAH record?

(yes, I’ve had a couple of pre-Thanksgiving drinks. Don’t worry, you guys will be in my novel someday and I will treat you with more respect than you think.)

Yes, 39 years is an endpoint … if you’re using a 78-year smooth. Get off it, Stokes.

bugs,

“Anyone else get a chill down the back of your neck? Anyone else feel insulted?”

I can understand why you would and I don’t.”

Yup, Che Guevara’s bosom buds had warm and fuzzies just like Bugs, that is, until they were offered the blindfold!!

Good luck with that Hope and Change bugs!!

lucia (Comment#24867)
Why are you picking just one comment?! Come on. Sheesh.
Because WUWT wrote a whole post on just that comment, as did Jeff Id (where you made your first OMG response), and others. But in fact in that latest WUWT post, many of the bolded comments are just versions of this same issue (including the one that references Osborn 2004).

Nick–
Come on now. Seriously. I mention Harry file as an example of a code that doesn’t simply calmly call out some sort of “adjustment” and link to a post at Antony’s containing dozens of lines from the harry file, and then you start going on and on about a quote in the “maps12.pro” file. The “HARRY_READ_ME.txt” includes lines like

“22. Right, time to stop pussyfooting around the niceties of Tim’s labyrinthine software suites – let’s have a go at producing CRU TS 3.0! since failing to do that will be the definitive failure of the entire project..”

and “getting seriously fed up with the state of the Australian data. so many new stations have been introduced, so many false references.. so many changes that aren’t documented.”

“Oh yeah – there is no ’supposed’, I can make it up. So I have ”

Then when I ask you why you discuss that one line, you say Anthony’s discussed that particular line a lot?

If that Harry code is real, it sounds like the programmer was hunting around and trying to create CRU out of whatever he could fish up. It doesn’t read like a guy with a well laid out notion based on a well explained journal article that tells people how to create temperature products from station data.

Freeing the code is the first step but the bigger problem is the complete unwillingness amoung the climate science community to accept that non-climate scientists looking at the code can have useful things to say.

My problem with this “free the code” mantra is that it came out of an attempt to cast doubt on the concept of global warming for political or ideological reasons, not out of any sincere interest in the science. I don’t believe that non-scientists looking at code will have much of anything useful to say. Some of them can’t even spell.

My comment appears to be held up in moderation. Perhaps because I sound too much like TCO? TCO and I are going to sue you over this moderation policy!!

greenaway (Comment#24879),

“I don’t believe that non-scientists looking at code will have much of anything useful to say. Some of them can’t even spell.”

What a bizarre comment.

Most here are in fact scientists or engineers, and most can spell a bit (though i’m not sure what difference that makes). Certainly many can understand and go through computer code and find errors/problems.

greenway

My problem with this “free the code” mantra is that it came out of an attempt to cast doubt on the concept of global warming for political or ideological reasons, not out of any sincere interest in the science.

I have no idea why you think this.

I don’t believe that non-scientists looking at code will have much of anything useful to say. Some of them can’t even spell.

One thing the HackCru files does show us is that many of the scientists can’t spell either.

“No one needs paleo to know the temperatures rose recently. The blade not the important part of “the hockey stick”. The important part is the shaft.

So, the finding that the shaft was wiggly repudiates ‘the hockey stick’.”

All he says is that he has more wiggles than other papers, the blade is still there, he thinks AGW is real.

Boris (Comment#24880),
“Perhaps because I sound too much like TCO?”
.
Na, unless you have odd fantasies about Sara Palin.

By gosh, I’ve got it! Here’s a simple seven step process that will lead to data and method transparency in short order:
1) Get a climate science journal to published one of your articles (preferably with an accompanying press release and RealClimate blog post)
2) Don’t provide the data and method transparency required for others to replicate your work
3) When approached politely by others for your data and methodology, ignore them or refuse outright
4) When confronted with Freedom of Information Act requests for your data and method, conspire to obstruct the request
5) If obstruction in step 4) above seems to have a significant chance of failure, delete your e-mails (and perhaps your data) and conspire with your colleagues to do same
6) Watch helplessly while persons unknown post your e-mail, data and methods on the internet (this is penultimate step in terms of difficulty since since you thought you’d deleted it all)
7) Go to prison and room with a very large, very hairy and very frightening man.
The beauty of this proposed process is that we’re almost there. The only thing left to do is step 7).
Simple
OA

“7) Go to prison and room with a very large, very hairy and very frightening man.”

Not going to happen. Worst case is “reassignment” or “retirement”.
Not saying that it shouldn’t happen, just that it won’t.

Here’s my problem: I agree with scientists being open with their data to as much a degree as is possible. Sometimes, there are legitimate restrictions on how open they can be, such as proprietary issues. I agree that without open access, replication may not be possible and so one of the pillars of the scientific method is undermined.

However, I believe what I see in the emails, is that much of the reluctance to provide CA regulars with the data is that they are perceived to have an agenda, a bias, and will use any error or misplaced bit of code as evidence of fraud in order to undermine the science. That is my perception and I think it might be the perception of the scientists in question. In my time in the field of science, my colleagues were quite happy to point out flaws in my work and offer critiques of my methods. This is considered part of the way science is done. You learn that very quickly with your first conference paper or submission to a journal. But it is done in the spirit of moving the science forward, improving your skills as a researcher, and it is undertaken by other scientists who understand the field and general approach.

What I see happening in this branch of science is very different from that, and in my view, it is solely because of the politics involved and the policy implications of the research.

Claims that climate science is a fraud and hoax appears to be BAU among CA denizens and other “skeptic” blogs that I have come across. Mistakes or missteps are held out as proof of bad intent and results in claims of fraud, hoax, lies, etc. and results in calls for resignations and firings and jail. This is a very threatening environment in which to work. It feels as if science itself is under attack and by whom? Non-scientists or those with a political objection to public policy that might come out of the findings of the research.

If this was less of a politically charged issue, say if it was instead about the mating habits of the lesser vole, I doubt there would be any auditing of the science or papers.

I don’t condone the actions of the scientists if they deliberately deleted emails to avoid FOI requests, or if they are manipulating the data to make it fit to some preconceived notions. My problem is that I am skeptical of the “skeptics”. I don’t feel I can trust them. I fear that many of them have ulterior motives of trying to delay and obfuscate action on climate change because they disagree with any government action, and are willing to do whatever they can to achieve it.

[...] Von Storch CRU Reaction [...]

“So, the finding that the shaft was wiggly repudiates ‘the hockey stick’.”

So it’s not a hockey stick shape. It’s a fuzzy long thing with a curve up at the end still. You wont’ be able to use it to play hockey with, but I really can’t see why that matters.

@SteveF

Yea, I know. There’s the way things are and the way things ought to be. I try not to get those two things confused in real life, but blogworld sometimes offers temptations than simply can’t be resisted

OA

Meanwhile, in the real world

http://scienceblogs.com/deltoi.....gnosis.php

The IPCC projections are once again being shown to be inadequate, with the models only just keeping up with reality.

Lucia #24878,
A problem with this stuff is that a whole lot of fuzzy items are advanced, with sweeping claims. When you look at each in detail, the subject gets changed.
At tAV, you said
Uses “corrected” MXD – but shouldn’t usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.
Oh common… This must be edited in. Must. Must. Must. I mean… come on. Someone actually put this in this way as a comment?!!!

You commented here on the WUWT post in much the same terms, so I assumed you were referring to the same comment and its many relatives. But no, we’ve moved on, so the topic is now Grumpy Harry.

Well, OK, what code was GH actually talking about? And what was his role at CRU?

Greenaway,

Claims that climate science is a fraud and hoax appears to be BAU among CA denizens and other “skeptic” blogs that I have come across. Mistakes or missteps are held out as proof of bad intent and results in claims of fraud, hoax, lies, etc.

There is a simple remedy. Release the code and data and be prepared to engage the truly scientific critics like SteveMc fairly by acknowledging legimate errors which will be found. This would build good will and credibility to deal with the things are are just misunderstandings.

The current approach where people like SteveMc are demonized as a corporate shills demonstrate bad faith and arrogance. Scientists who act like that should not be surprised to find that every mistep results in accusation of fraud – they deserve it.

The issue of transparency and reproducibility is not about public funding (although that serves as another reason why everything should be available). It’s about public policy. No study should ever be used to influence public policy that fails this basic test. Even if it is privately funded.

If the study can’t be checked by others in every detail, it should never, ever be considered anything more than an unsupported opinion — an untested hypothesis. And contrary to the team members in the phil jones e-mails, other people doing their own studies is not going to cut it! If two researchers make errors and the public cannot check their work, why should two mistakes equal sound science? That’s just stupid. Or climate science.

There is a simple remedy. Release the code and data and be prepared to engage the truly scientific critics like SteveMc fairly by acknowledging legimate errors which will be found. This would build good will and credibility to deal with the things are are just misunderstandings.

Lol. The man who can’t start an ‘audit’ without a snarky remark and doesn’t publish. Highly scientific.

@bugs.

Heh. The real world. You are one funny guy. Could you please give us links to the peer revie… Uh, wait… To the CREDIBLE literature that supports the assertions in the linked puff piece?

Thanks,

OA

Some fields do require data and code. Here is teh replication policy of Econometrica, one of teh premier economc theory/econometrics journals.

2.3. Replication Policy

Econometrica has the policy that all empirical, experimental and simulation results must be replicable. Therefore, authors of accepted papers must submit data sets, programs, and information on empirical analysis, experiments and simulations that are needed for replication and some limited sensitivity analysis. (Authors of experimental papers can consult the more detailed posted information regarding submission of Experimental papers.)

This material will be made available through the Econometrica supplementary material web-page. Submitting this material indicates that you license users to download, copy, and modify it; when doing so such users must acknowledge all authors as the original creators and Econometrica as the original publishers.

At the same time the editors understand that there may be some practical difficulties, such as in the case of proprietary datasets with limited access as well as public use data sets that require consent forms to be signed before use. In these cases detailed data description and the programs used to generate the estimation data sets must be provided, as well as information of the source of the data so that researchers who do obtain access may be able to replicate the results. This exemption is offered on the understanding that the authors made reasonable effort to obtain permission to make available the final data used in estimation, but were not granted permission. We also understand that in some particularly complicated cases programs may have value in themselves and the authors may not make them public. Similarly, there may be compelling reasons to restrict usage, and if we agree we will post a notice on the web site regarding such restrictions.

Requests for an exemption from providing the materials described here, or for restricting their usage, should be stated clearly when the paper is first submitted for review. It will be at the editors’ discretion whether the paper can then be reviewed. Exceptions will not be considered later in the review and publication process.

greenaway (Comment#24879) November 25th, 2009 at 9:59 pm
“Freeing the code is the first step but the bigger problem is the complete unwillingness amoung the climate science community to accept that non-climate scientists looking at the code can have useful things to say.
My problem with this “free the code” mantra is that it came out of an attempt to cast doubt on the concept of global warming for political or ideological reasons, not out of any sincere interest in the science. I don’t believe that non-scientists looking at code will have much of anything useful to say. Some of them can’t even spell.”

Well, let me explain where “free the code” came from since I used it so liberally in getting hansen to free his code. You can go back to 2007 at RC, probably where I first used it.

http://www.realclimate.org/ind.....process-2/

The “motive” you ascribed to my request ( joined by judith Curry in the 1934 thread) doesnt have anything to do with undoing global warming. Moreover my motives for requesting the data and code are in someways unknowable.

Now back in 2007 Timothy Chase and others at RC mis identified me as Steven W Mosher ( its happened several time) and assailed my motives based on that stupid mistake. If you want to know my motives? I’m an open source advocate. Go figure.

You see one reason to free the code is to REMOVE these unscientific appeals to motive that we all engage in.

PS. after hansen freed the code mistakes with GISSTEMP were found and corrected. Everyone forgets that.

“PS. after hansen freed the code mistakes with GISSTEMP were found and corrected. Everyone forgets that.”

I remember it all as a storm in a teacup.

Steven Mosher

Steven, Gavin has announced another paper TODAY called The Copenhagen Diagnosis.

Same old crap. Sea level rising faster than ever, poles (both) melting faster than ever, temps rising faster than ever. Quote ” I won’t comment on it because three of us here (RC) wrote.

Still peddling the same old BS. What has he got for brains??

lucia (Comment#24764)

Lucia

It does not become overwhelming if you do you coding professionally. It is very easy. There are many programs on the market for version and release control. Simples;; )

It’s interesting that Steve Mc has said on several occasions that he suspected that Jones’ refusal to disclose the CRU code was because it was trivial, despite attracting large funding. Now it seems it might be because it’s an embarrassment.

BTW, Nick Stokes, it’s clear that Harry was working on the code that produces CRU TS 3.0, as he says so.

The “motive” you ascribed to my request ( joined by judith Curry in the 1934 thread) doesnt have anything to do with undoing global warming. Moreover my motives for requesting the data and code are in someways unknowable.

What is your motive? You don’t even know? Who are you?

Now back in 2007 Timothy Chase and others at RC mis identified me as Steven W Mosher ( its happened several time) and assailed my motives based on that stupid mistake. If you want to know my motives? I’m an open source advocate. Go figure.

Yes, I read about the mistake. From what I understand, it amounted to no big deal. Nothing changed that has any consequence in terms of the science. But you and others made a big stink about what ended up being nothing. You did have quite a political affect, whipping up conspiracy nuts who are prone to such tempests, by casting undue suspicion on the work and those involved. Which, even if you don’t know your own motives, I suspect was your goal.

“followed by a clear warming trend in the past two centuries” Bugs #24862

So what you’re saying is that von Storch said that the current global warming started well before global industrialisation and its resultant rises in CO2 output?

No wonder the hockey team don’t like him much!

bugs (Comment#24903)
November 26th, 2009 at 4:15 am

I remember it all as a storm in a teacup.

I thought all these metaphors pooh-poohing this stuff were supposed to be based on mountains.

Ahhh…

http://www.flickr.com/photos/ranakwil/270856934/

OzzieAardvark (Comment#24897) November 25th, 2009 at 11:59 pm

@bugs.

Heh. The real world. You are one funny guy. Could you please give us links to the peer revie… Uh, wait… To the CREDIBLE literature that supports the assertions in the linked puff piece?

The sea level rise charted in the link is taken from Cazanave et al. 2008. Full paper here:

http://sciences.blogs.liberati.....C_2008.pdf

Greenway,

Why to alarmists have such thin skins? Who cares if a bunch of yahoos exaggerate the significance of problems or errors? I assume you are all adults.

I do hope that the releasing of code does not become a substitute for a correctly specified method.

It is the method that guides the reader as to what has been done. If the method is clear and complete it should be possible to reproduce the analysis and produce the same results. Having the code would be handy but, providing it is the code actually used, running it will automatically produce the result, so unless you verify the code against the method you are no wiser as to whether it is performing the analysis as claimed.

I just hope that excessive emphasis on code does not lead to papers adopting an approach like this:

Method Section: For the data handling methodology (see SI)
SI: For the data handling methodology (see code)

Given you have the actual method. What is then important is the data, and the actual results (not just a graphic), without these you are stumped.

It should be possible to replicate vased on the data and the method. In some senses it would be better not to use the code provided but to start again from the method as otherwise it is not a replication in its broadest sense.

I suspect that papers are and will continue to be reviewed with respect to the method, not the code.

If the problem is that we no longer trust the method and hence must have the actual code then that is a problem in the quality of current method sections.

Also I can see no reason why code produced for research purposes should be pristine. It is a tool that only needs to do precisely what the method implies and potentially just run once or a small number of times. I should hate for papers to be unnecessarily delayed just because the code is deemed not to be pretty enough.

The Nature “trick” is not a code trick, it is a method trick. If it had been declared in the method proper or as extended in SI available at the time of publication, the code would have been irrelevent. And best of all you would not have to be a programmer to understand what has been done.

If a paper is reviewed and then published on the strength of a method that is not the one actually used to perform the analysis then the publisher and also the reviewers have got a serious grievance with the authors. I really do expect reviewers to trust the written method and not have to send off the code to some expert so that the actual method can be reconstructed.

I am only aware from first hand knowledge of one paper where the advertised and actual methods diverged (the results also diverged between using the described and actual methods). But then I have only checked one. It is kind of dispiriting when this goes on.

If they had provided the code, I could have run that, got the same result, and been none the wiser. As it happened they published results (SI possibly at a later date) at an interim stage and it was clear that the declared method could not have been followed.

Now I realise that the published method is hand on heart stuff, if the author writes what has been done it should be gospel. That is the way it ought to be. If authors are caught misleading reviewers, publishers, and the readership by saying one thing and doing another then they ought to be taken down a few pegs and not trusted in the future.

I think that releasing the code would do no harm, providing that it is only judged on whether it tallies with the method, please do not let us have beauty contests.

But publishing the code is no substitute for a full and honest disclosure of the methods.

Alex

Nick–

A problem with this stuff is that a whole lot of fuzzy items are advanced, with sweeping claims. When you look at each in detail, the subject gets changed.

Good gracious Nick! You have a very odd notion of what constitutes changing the subject. In a comment at TAV two days ago, I happened to quoted an example of a specific damaging comment when saying things must be edited in? So, in comment here and now, I’m not allowed to be referring to the full range of comments in the “Harry” file? And your going to a quote that is not in Harry is staying on the subject because you can fish up a time several days ago in a different conversation where I happened to discuss a quote in another bit of code?!

Bugs–

So it’s not a hockey stick shape. It’s a fuzzy long thing with a curve up at the end still. You wont’ be able to use it to play hockey with, but I really can’t see why that matters.

It matters what terms you use to say what was shown. If the point that you want to make is that even though the hockey stick was killed, the theory of AGW is still intact, that’s fine. If you want to say Von Storch restored the hockey stick, that’s false. Von Storch showed the reconstruction methods used to create “the hockey stick” were poor. In more inflamed political language, he showed it was broken.

The only issue he and McIntyre have is the level of credit M&M should get for breaking it. I think VonStorch is being rather grudging in credit– even if from some obscure math reason,after they found a flaw, he went looking and found an even bigger better flaw in the hockey stick than M&M found.

Alexander–

I do hope that the releasing of code does not become a substitute for a correctly specified method.

That’s my concern.

I think one problem has been the brevity of papers has left methods inadequately described. In some cases, it is appropriate to have many nit-picky details clarified by code. But in many cases, the methods are simply not being described. This is happening even with some nearly “brand new” methods and sometimes in short papers where the entire point is to describe a new method (say how to treat endpoints when smoothing.)

It’s ludicrous to say this can be “cured” by providing the code. It should be cured by insisting on people describing important methods that make a different to the result in the papers themselves.

lucia (Comment#24878) November 25th, 2009 at 9:57 pm says

If that Harry code is real, it sounds like the programmer was hunting around and trying to create CRU out of whatever he could fish up. It doesn’t read like a guy with a well laid out notion based on a well explained journal article that tells people how to create temperature products from station data.

To me Harry_READ_ME.txt looks like a well intentioned programmer that has inherited a complex, poorly documented, somewhat defective set of code and is trying to update it to a new version.

I was managed a project where the code and design just kind of morphed to the point where it was incredibly messy and unreliable. It was very painful at the time, but I decided to scrap it all and start over. We had a tremendous reservoir of knowledge on what NOT to do the 2nd time around.

Starting over from the beginning was one of my best management decisions. Ever.

————————————-

My prediction is that one of the long term consequences of “Climategate” is that the CRU (or another group, such as the Hadley Centre of the Met Office), is going to go back to basics and start over with the raw station data and the metadata and build from there all the way to adjusted, homogenized, gridded data. This may take 2 years or even 5 years, but it is something that really needs to be done.

I fully appreciate that you guys are very concerned with full disclosure of data and models, which of course is an important topic.

However, this was of only one of the ideas of Van Storch’s blog, where he said:

“Another conclusion could be that scientists like Mike Mann, Phil Jones and others should no longer participate in the peer-review process or in assessment activities like IPCC.”

I am more concerned with the IPCC proces in general, and find thats it needs to be redesigned. And I am looking for your views on that.

Simon Evans (Comment#24789)

“Which field do you want to consider? Evolution is evidently a load of baloney – where’s the data, where’s the code?”

Hmmm. Good question. Can you say the word “theory?”

jae (Comment#24929)

I guess one could say the “code” is in the genes, eh?

jae,

Anyone who resorts to arguing about ‘Evolution’ when they are supposed to be arguing ‘Global Warming’, clearly has run out of things to say about ‘Global Warming’.

Andrew

Karl–
I’m not sufficiently familiar with the details of the IPCC process to have an opinion in general. However, getting people who work hard to impose a view and references to peer reviewed articles that defy their POV would we wise regardless of how the particular rules are constituted.

Sometimes, the rules as written read fine, but they are disregarded by those who are expected to follow and/or enforce them.

Don’t be daft Andrew – I wasn’t arguing about Evolution, as you very well know. Let’s stick to topics rather than personalising it, eh? Anyway, sorry to disappoint you, but I haven’t run out of things to say!

It appears to me that several of the regular warmers here have been gobsmacked pretty badly by the leak at CRU, and as a result, are wincing and nitpicking. Admit it, guys, some very problematic behavior has been revealed, and efforts to cover it up will simply highlight it. You are acting just like the MSM, which STILL doesn’t seem to get this, as evidenced by how much their character assinations are simply enhancing the popularity of such “horrible conservatives,” like Sara Palin and Glen Beck/Fox News… And Climategate!

This looks like an interesting statistical technique. Who needs to mess with code and numbers when you have a straightedge and your eye? Can anyone explain what’s going on? Statistical graphing packages don’t have a way to shade areas? And why are they aiming?

Found in Alleged CRU Email – 1200010023.txt and Alleged CRU Email – 1200003656.txt [ from search on the word cheat]

10/01/2008, Peter Thorne wrote:

Grey shading is a little cheat from Santer et al using a trusty ruler.
See Figure 3.B in this paper, take the absolute range of model scaling factors at each of the heights on the y-axis and apply this scaling to HadCRUT3 tropical mean trend denoted by the star at the surface. So, if we assume HadCRUT3 is correct then we are aiming for the grey shading or not depending upon one’s pre-conceived notion as to whether the models are correct.

Who wants to document the use of rulers in their papers. Amongst your friends and supporters sure but ….. well.

In Alleged CRU Email – 1200003656.txt this quote of Phil Jones seems salient to question of brevity vs. tedious discussion of technique in papers.

“the text can’t be too long. I would favour copious captions, and even an Appendix, to get the main points across quickly.”

Simon,

You brought up Evolution as comparable to Global Warming in some way, not me.

Anyway, I look forward to all your future comments about Global Warming and otherwise!

Andrew

lucia (Comment#24915)
“And your going to a quote that is not in Harry …”
OK, it’s not in Harry, but it stars in multiple roles in the WUWT post you linked to. This is the fuzziness I complained about. So you’re not talking about that, and never were. It’s always something else, and very wrong, but you never say why. So what were you talking about?

=Nick..

No you’re not talking about that, and never were. It’s always something else, and very wrong, but you never say why.

Oh… let me see what did I say:

Well..but you have to admit if that Harry code discussed at Anthony’s blog is real and not faked, it isn’t a matter of just calm note saying “adjustment”.
(I’m inclined to believe that file was juiced. I mean… OMG!!!!)

What I meant is that the types of things people are complaining about aren’t about a mild mention of an “adjustment”. The code contains boatloads of panic comments discussing disorganization in databases, lack of specificity in what choices the person banging out code is intended to make, and things like “What the hell is
supposed to happen here? Oh yeah – there is no ’supposed’, I can make it up. So I have ”.

The reason why Jones may not have wished to release the code is not fear over nit-picky complaints over minor things. It is that it would reveal the total disorganization of the code.

Or, the Harry code could have been edited by “The Hacker/Whistleblower”. Who knows? But it’s contents, if real, should be embarrassing to Jones. If he knew his codes looked like this, then we can understand why he would not wish to release them in the sense that we can understand his motivation.

I’m not sure you all get what I am aiming at. Nothing less than “recompiling science.” With electronic documents this becomes possible. I think of lucia’s simple example: I computed the sum of integers between 1 and 10 and the result was 55. And for this she suggests that no code is required. But now think of a way I can make the document BE THE CODE. Where that sentence is linked to the code. Think of executable latex. Now lets imagine that lucia had made a minor mistake in her procedure. Where she really didnt want to sum between 1 and 10, but rather 1 and 11. and lets suppose that she used 55 in subsequent calculations. It would be a great thing for me to be able to open her document as a reviewer, spot the error, change the number from 10 to 11 and have the figures for the document instantly update. Of course the text and conclusions may change. Perhaps her table of correlations shows different numbers. And her text which says P>.95 suddenly gets transformed to say P>.8.

Everyone says such an idea is silly. In 1988 I I left work in defense for two brief weeks to take a job as the head of the computing for humanities lab at UCLA. They put a MAC on my desk, arrg I was used to an SGI workstation. But I found this cool little thing called hypercards. And I thought wouldn’t it be cool if I took a book or document and used this linky thing to document my research on a book. You know so if you click on a word in the text you get transported to other texts. Sadly, nobody in the lab thought that was a cool idea so I quit and went back to defense where I plotted the destruction of idiots. hehe. Anyways, the same thought occurred to me as I wrote analysis reports for experiments. What if….

http://sepwww.stanford.edu/dok.....ible:seg92

http://sepwww.stanford.edu/pub.....index.html

an extraodinary extremely professional posting at CA
http://camirror.wordpress.com/.....t/#more-75 no wonder the AGW are trembling. Hope some jailtime its extremely serious

Ok last linky

http://kc.vanderbilt.edu/quant.....alyses.pdf

Steve–

I computed the sum of integers between 1 and 10 and the result was 55. And for this she suggests that no code is required. But now think of a way I can make the document BE THE CODE. Where that sentence is linked to the code. Think of executable latex.

I didn’t say the idea that one might some day be able to easily fulfil your stringent standards for documenting even the most simply numerical claims is silly. It’s fine to be forward looking.

But first: it’s ridiculous to insist that everyone must fulfill your visionary dreams for documenting any and all codes in every single form of communication no matter how informal simply because some day it might be possible do do so easily using executable latex (which I believe does not yet exis) . Even non-executable latex is a specific markup language, which while splendid as a markup , makes the statement 1+1=2 no more true than if it were written in plain text, crayon or any other form.

I think it would still be ridiculous to suggest that I couldn’t say the sum of the integers from 1 to 10 is 55 without also turning my narrative into an excutable code or providing a code of some sort. It is particularly ridiculous to start suggesting that you think it would be easy if only one used “So and so’s favorite computing language B”.

Communication of information and ideas should not, and cannot be, held hostage to somebodies notion of the best forms of codes, documentation etc. The standard for openess is a) can interested people gain access to information required to check thing or learn more in some reasonable period of time. (For many involved papers getting the code within a week should fall within the ‘reasonable’ window. ) b) are the standards for excluding some people from some information remotely reasonble.

For example: for classified work, only those with security clearances and need to know get access. The stuff can still be science, but the program can and should make sure that disinterested parties check things before using. For work with real IP, companies get protection etc. But all this is clearly documented in advance and not cooked up willinilly by the Principle Investigator.

Making overly broad ‘rules’ about “freeing the code” and thinking you can apply them to informal of communications is silly. If you want to say that you will suspend judgment because your criteria aren’t met, then do so. Many people suspend judgment on informal communications for all sorts of reasons. Meeting your criteria on codes isn’t going to change that. For that reason, people aren’t going to meet it.

Moreover, your trying to insist on some overbroad application just makes it more difficult to communicate why the principle of providing code is reasonable in the instances where codes should be provided. Code for GISSTemp should be provided. It’s a shame getting it freed required nagging from Congress. Code for CRU should be freely available. It wasn’t because, basically Jones ‘don’t wanna give it out’. But that these codes should be “freed” do not support claims like blogs need to provide turnkey codes to support the authors claim that 1+1=2.

jae (Comment#24935)

If you threaten me with Sarah Palin I’ll confess to anything .

I’ve said in enough comments now that I think there are matters that need to be investigated, so I’m certainly not interested in ‘covering up’.

I think the consequences of the emails being revealed is significant in terms of the effect upon public opinion. I think the past consequences of what the emails reveal is very likely to have been absolutely trivial ‘in global terms’, though not necessarily trivial for some individuals.

I don’t see anything good for science in the ‘questionable’ emails but I also don’t see anything good for science in much of the reaction to them. Proposing higher standards is good, but seeking to make propaganda capital out of this is not, and that dismays me far more than the emails themselves.

Simon Evans (Comment#24948)
I think the past consequences of what the emails reveal is very likely to have been absolutely trivial ‘in global terms’

I fear not, the additional inormation now appearing:

http://wattsupwiththat.com/200.....m-tactics/

http://wattsupwiththat.com/200.....peratures/

http://wattsupwiththat.com/200.....icial-one/

Is begining to undermine the validity of the global temperature measurements, when added to the highly dubious nature of some of the “High quality” weather stations it would appear that we actually “know” precious little about the global temperature trends

Arhur Dent (Comment#24953)

Arthur,

Let’s have a little scepticism, please!

Your links:

1. This is Pielke Sr’s objections to his experience with the CCSP committee, whose report was published in 2006. Pielke’s own paper expressing his reservations is freely available (from his copious links to it) and has probably been read by more people thatn have read the CCSP report. Regardless, this has nothing to do with the purloined emails. Btw, though, you can see here that the CCSP report included Jihn Christy and Roy Spencer as authors! -
http://www.climatescience.gov/.....efault.htm

2. This also has nothing to do with the emails. Your link is to a story related to a 1986 article! The station coverage in Australia has improved since then, as you can see from this:

http://www.sciencemag.org/cgi/.....7/5324/364

3. This also has nothing to do with the emails, but here is the response statement from NIWA:

http://www.niwa.co.nz/news-and.....ature-rise

Let us know what issues you have with that.

You read WUWT without, apparently, any scepticism? Gosh.

Arthur,

My response to you seems to have been caught by the spam filter. I suggest you should be sceptical of what you read on WUWT.

Simon Evans (Comment#24948)

Simon, maybe you are still in shock. Like the MSM, you seem to be trying very hard to ignore this undeniable scandalous travesty of all that science stands for. Good luck with that. There is no doubt that a very important chunk of AGW “proof” (the hockey stick BS) has been, FINALLY, firmly and thoroughly trashed. These snakes did not get away with “doing away with the MWP!” And as long as folks keep the MWP, RWP, etc. in mind, it is really difficult to have too much confidence in AGW–any warming could easily be due to natural variation. And especially so, now that warming has abated for 12-15 years!

And the temperature records are even more questionable now than they were in the past. Has it really warmed in modern times?

It’s fascinating that very little of what is shown in the emails has not already been demonstrated by CA over the last 3 years. But now, we have the type of delicious scandal that will FORCE the whole world, even believers, to finally pay attention.

jae (Comment#24956)

You have started to shout, jae – it’s usually a bad sign.

I’d be interested to know what you think has happened to the effect of GHG absorption during this period of “natural variation” you propose.

Has it really warmed in modern times?

Gosh, that really gets to the heart of it, eh?

Lucia,Color me silly. people called me silly when I suggested creating hyper books where the text linked to commentary on it. people called me silly when I suggested that the 3D graphics we had on SGI would someday be on a PC and then someday on a phone. People called me silly when a bunch of us created a handheld device that held all your music. It’ll never sell. Color me silly. But let’s separate my silly dream from the basic requirements for something to be considered science.

The basic issue I have with your suggestion is the element of subjectivity. If you go back to the origins of this debate over GISSTEMP ( the 1934 thread on RC) you will see gavin REPEATEDLY arguing that the paper definition was enough. Even when I quoted the paper and asked him to explain a very simple paragraph on the exclusion of 5 northern california stations, he REFUSED to admit the obvious. The words did not describe the algorithm. Not even close. You have seen how some people refuse to admit the obvious. You’ve been frustrated by it. My solution to this obstinance is to bully for all the code. It’s come to that.

And remember the paper is not the research. The paper is the advertisement for the research. The SI should be the center not the margin.

happy thanksgiving.

Re: steven mosher (Comment#24959)

So are you or aren’t you saying that the code used to compute sum on 1 to 10 must be published along with the science?

For many in the sciences, the statement, along with perhaps a few lines of intermediate derivation, is the “code.” Should it now be “executable” in LaTeX (in the sense that it might be executable in Maple, for instance) to be adequate documentation?

Oliver,

I am suggesting that all the steps taken in research be published.
basically show your work.
Lucia’s example of trivial computations cuts both ways. On one hand it looks like one should NOT have to provide the code for 1+1=2. But, if in the course of research you wrote a line of code
to compute 1+1=2, then by all means that code must be delivered. Its already written. Just package it up and deliver it.

But isnt that code redundant to the text ( her argument that the text is the computation) Yes, it’s redundant. Communication is full of redundency: it improves accurate transmission.

Now if you wrote under methods:

” The sum of the integers 1 to 10 was computed mentally and the result of 55 was obtained. ”

Then I would agree that the code for this need not be supplied. Since none was written. And the instructions for scientific behavior are complete: compute the sum mentally. But if you write:

” the numbers 1 through 10 were entered into an excell spreadsheet and the column was summed. The result 55
was used in subsequent calculations”

Then I should like to see that excell spreadsheet. And you should provide it. because you can do so readily.

Steven,

While I find your principle of documentation admirable, like all things it can be carried to absurd lengths. I don’t actually need to know whether the result was calculated mentally or in Excel; alI I really need to see some notation that says where sum(1..55) was used and I can verify the value myself if need be.

If it really were carried to these extremes, every academic paper would grow to the size of a textbook + answer key.

“So what you’re saying is that von Storch said that the current global warming started well before global industrialisation and its resultant rises in CO2 output?

No wonder the hockey team don’t like him much!”

There was an ice age ending. The warming has continued well past that stage where solar was the main driver for a while.

My bad, not an ice age, a period of cooling.

…the issue of whether or not they belong to some insider group may be of some historic or sociological interest, but it is utterly irrelevant to science. It is also irrelevant to assessing whether they contributed to science.

It is relevant if nothing comes of it. Only time will tell if their work stands up to the test of time. Ultimately, a few lines of code, a line of data misplaced, coupled with many more allegations and insinuations doesn’t amount to much in my estimation. It does hold a lot of water with CA supporters, but then, they are prone to it in the first place.

It is very significant when one is considering the politics of the whole affair and its meaning and importance in a larger historical sense. Being part of some insider group with a particular interest –financial or ideological — in the outcome and implications of scientific research does matter, very much, in understanding their part in this. That is what I suspect when I consider motives, given the backgrounds of the Mssrs. McIntyre and McKittrick. AFAIK, they both have ties either to corporate interests that face potential losses if government action is taken on greenhouse gasses and/or ties to a particular political ideology that is anti-regulation.

If you can’t produce serious peer reviewed research in reputable journals that casts doubt over the science of global warming and convinces other scientists that it is erroneous, then the next best thing is to hack away at what scientists have already published, piecemeal if necessary, and failing that, hack away at their reputations.

Casting unwarranted public and political doubt on the science of global warming by picking it apart in ways that ultimately mean nothing to the larger question stands to delay political action that, if the “warmers” (as you all like call them) are right, and if their predictions are right, will be eventually very damaging.

I assume this stupid point is of the “talking points” variety and you will continue to repeat it like a mantra. Repeating it will not turn it into anything meaningful.

Stupid? Your opinion, I suppose. To me, it bears repeating because many deniers are holding up McIntyre’s and McKittrick’s work as somehow “breaking” hockey sticks and disproving global warming. Neither of which are true.

“That is what I suspect when I consider motives, given the backgrounds of the Mssrs. McIntyre and McKittrick. AFAIK, they both have ties either to corporate interests that face potential losses if government action is taken on greenhouse gasses and/or ties to a particular political ideology that is anti-regulation.”

This is pure projection, and of course mostly suggests that (a) you have a tie to interests that face benefits if unnecessary action is taken, and (b) you have ties to a political ideology that is pro-regulation.

Close enough?

LKucia I think you will need to re-visit this whole blog and its content refer to latest posting at CA and WUWT re data analysis. It is a complete fraud without any doubt. All the temp data (except R Spencers) that you have relied on to put up your graphs is based on this.
http://wattsupwiththat.com/
http://wattsupwiththat.com/200.....e-decline/
NONE of the data from these organizations (HADLEY ect) can be trusted and all the publications based on it should be withdrawn immediately

To Simon,

I take it at heart you are a believer whose faith is being tested, hence the seeming interest in the traditional scientific method and holding back any snark you might feel need to release. But your game is getting old here, you are not going to change. So let’s get it over with! Write down here what would be the most awesome, damning statement of denier stupidity that you could dream coming across to allay your soul. I will cut and paste it under my own name. There, you will have what you are fishing here for.

Perhaps what is required is the climate science equivilent of medicine’s epidemiology. Having suggested that, it’s certainly interesting to investigate how medical researchers view that discipline – ask a surgeon about his own experimental technique and his epidemilogical evidence that it’s effective and you’ll likely get a comment along the lines of “I don’t need no stinking epidemiologist to tell me it works!”, while asking the same surgeon about someone else’s experimental technique will likely result in the first question being “Do you have the epidemiology on that?”. Oddly enough, almost none of them see the irony in that.
It certainly seems to me that climate science is also blinded by this sort of confirmation bias – it’s human nature, so that’s not a put down. It also seems to me that those such as our gracious host, as well as SteveMc and others, are performing the epidemiological equivilent – it’s just that it’s yet to be formalised and agreed to be a useful practice. And The Team, as SM calls them, are playing the role of the experimental surgeon in my analogy above – they don’t need no stinking statistician to tell them they’re wrong!
Just food for thought…

greenaway(Comment#24966):

The insinuation that Steve McIntyre has corporate ties is unfounded and silly. It’s also kneejerk and sophomoric to assume that anyone who does not accept CAGW hype must have close ties to some evil corporate interest.

Ideology, political influence, media attention in pursuit of access to taxpayer money has obviously corrupted the folks whose email we have been reading. Your side has the money-power-bias problem, not mine. Restating True Believer pap about true science versus corporate perfidy is a waste of bandwidth.

You bloviate about peer reviewed science when the issue is the extent to which the publication and peer review process itself has been hijacked by the very people you defend. The Consensus of Omniscient Peers is no longer operant as an argument. Do you get that? Do you have the slightest clue what has been discussed in these threads? What has happened in the news?

This is pure projection, and of course mostly suggests that (a) you have a tie to interests that face benefits if unnecessary action is taken, and (b) you have ties to a political ideology that is pro-regulation.

Projection is when one attributes motives to others that one secretly harbours oneself. Since I’m a professed “warmer”, you are mistaken.

It’s actually called deduction, which involves reasoning based on logical premises.

As in :

- “Human actions have a motive that induces them”

- “A is critical of global warming science and is active in searching for flaws in global warming research”

- “A must have a motive for his criticism”

- “Global Warming science suggests that anthropogenic CO2 emissions are causing the climate to warm”

- “One way to limit warming is to curb anthropogenic CO2 emissions”

- “One means of curbing anthropogenic CO2 emissions is government legislation”

- “Libertarians are against government legislation to limit CO2 emissions.”

- A is affiliated with a libertarian think tank which has come out against government legislation to limit CO2 emissions.”

Conclusion:

- “A’s criticism of global warming science is the result of A’s affiliation with libertarian ideas and rejection of government legislation to limit CO2 emissions”

It is entirely possible that A’s rejection of global warming science is not due to his libertarian ideas, but it’s not likely. A’s libertarianism would tend to make him suspicious of any government regulation of industry or the economy except for that which makes it more free. A would be ideologically inclined to reject intervention and thus, would tend to be suspicious of global warming science since one of its logical premises is to regulate national/industrial emissions of CO2.

I don’t mind when people oppose things based on ideology as long as they are clear and upfront about it. It’s when they hide their opposition in the guise of “skepticism” and brand their attacks as being done “in the interest of science” or “code freedom” that I balk.

Instead of attacking the science, why not make a sound compelling argument against government regulation and its ills and offer sound alternatives? Of course, there is no really good solution to the problem of CO2 emissions and global warming if one does take that position.

The solution, for those who have bad faith, is to attack the science rather than the real source of their opposition.

Lucia has suggested that the politics have no bearing on the science, but that is not possible to claim any longer. The whole science of global warming has become politicized because it has so many implications beyond the science itself, especially because certain sectors of the economy may be / will be inordinately affected by any climate legislation. Because it is a global problem, it will take global solutions. Libertarians especially are against the notion of global governance of any kind. I note that many of the most vocal opponents of global warming science and most committed “auditors” appear to be libertarian-leaning from their pronouncements.

Steven Mosher,

they will burn you at the stake before they give up the methods developed to obfuscate and hide their work and errors!! ;>)

You bloviate about peer reviewed science when the issue is the extent to which the publication and peer review process itself has been hijacked by the very people you defend.

Yes, the whole Climate Research / Soon and Baliunas paper affair served to show that peer review is not sufficient to ensure that good science is published and makes one highly suspect of certain journals and editorial boards. One can understand climate scientists’ concern about the health of peer review and publication and desire to keep it rigorous.

Speaking of hijacking of peer review and publication, I note that a number of “skeptics” attempted to get published but their work was found wanting in scientific terms and so instead, they publish on blogs or in journals that are not recognized by the scientific community. The few skeptical articles I have read have really been inadequate. If skeptics could just produce credible scientific work that challenged the science of global warming, I’m sure they would be seen as heroes. Most people who currently accept the global warming science would be relieved to discover that it’s all a big mistake, with the exception of the cranks.

greenaway (Comment#24966)

“If you can’t produce serious peer reviewed research in reputable journals that casts doubt over the science of global warming and convinces other scientists that it is erroneous, then the next best thing is to hack away at what scientists have already published, piecemeal if necessary, and failing that, hack away at their reputations.”

LOL. You must be new to this debate. For one thing, the “refutations” are already in the “peer-reviewed literature.” And the “hacking” sounds you hear are the crooked scientists, themselves, hacking an ever deeper hole in the ground around them…. Not only have these “peer-reviewed scientists” already confirmed in the leaked emails virtually EVERY criticism that has been offered, but now they CONTINUE to tell more lies. It is truly pathetic, but also quite enjoyable, in one sense…

Oliver

If it really were carried to these extremes, every academic paper would grow to the size of a textbook + answer key.

And worse, given what is expected in delivered materials, there will inevitably be calls that all these things be copy edited to make sure there are no spelling or grammar errors in the comment fields! Some one will want to apply a style guide etc. And all this will be applied to loads of stuff whose anticipated audience is near zero in the case of most paper!

StevenMosher– Its well and good to give the example of GISTemp– and methodology which clearly could not be documented in a journal article. The remedy for that would have been for NASA GISS to remember that they have the option of supplementing their thin page limited journal articles with big thick NASA reports. Similar things are published at all National Labs. Also, given GISSTemp is a production code that is used monthly, it clearly should have been made available.

But this still is no argument for the notion that a blogger must upload and archive an excel spread sheet if they happen to write “the sum of integers from 1 to 10 is 55″ and happened to compute that with a code rather than just scrawling out the formula on a napkin and doing the 7th grade math problem. There needs to be some sense of proportion, and some recognition of which codes “must” be provided, which would be nice to provide and which just don’t need to be provide to anyone at all.

Greenaway,

“Speaking of hijacking of peer review and publication, I note that a number of “skeptics” attempted to get published but their work was found wanting in scientific terms and so instead, they publish on blogs or in journals that are not recognized by the scientific community.”

I see, only those supporting AGW are not found wanting in scientific terms?

I don’t think I need to bloviate any more on your empty rhetoric.

Greenaway- “A’s criticism of global warming science is the result of A’s affiliation with libertarian ideas and rejection of government legislation to limit CO2 emissions”

Or perhaps A simply values the truth. Years ago CS Lewis coined the term “Bulverism” for your line of argument which rests on the premise that it is unnecessary to rebut an argument by showing that it is based on false premises or faulty logic if you can impeach your opponent’s reasons for making it. Hence truth is irrelevant and motive is all on your view.

Not only have these “peer-reviewed scientists” already confirmed in the leaked emails virtually EVERY criticism that has been offered, but now they CONTINUE to tell more lies.

I read all the emails. We must have a different understanding of them. To me, the emails provide evidence that deniers have succeeded in politicizing climate science through their personal attacks and innuendos and scientists have responded by trying to defend it and their work. I have read their responses to the hacked emails and they provide explanations for the out-of-context emails and quotes. I note that the hackers only released a certain number of emails and documents — perhaps if we had them all, we’d have the context and not just the ones cherry-picked to try to make them appear suspect.

Or perhaps A simply values the truth. Years ago CS Lewis coined the term “Bulverism” for your line of argument which rests on the premise that it is unnecessary to rebut an argument by showing that it is based on false premises or faulty logic if you can impeach your opponent’s reasons for making it. Hence truth is irrelevant and motive is all on your view.

I find it amusing that people who reject my conclusions about motive on the part of skeptics are not disinclined to draw conclusions about the motives of climate scientists. I’ve seen skeptics claim that climate scientists on “the team” are motivated by greed, ideology, grant money, fame, and that they fudge data to make it appear that there is warming where there isn’t, select trees to create curves that conform to expectations, or adjust temperatures to show warming, etc. What’s sauce for the goose.

Greenaway,
People doing unethical things always *believe* that they are being ethical. So it does not make a difference if Jones/Mann and co *believed* the needed to fight back against those horrible sceptics. The fact is they engaged in unethical activity that corrupted the peer review process when the blackmailed a journals and editors for publishing papers they did not like.

People doing unethical things always *believe* that they are being ethical.

Like people hacking CRU’s server and stealing emails and documents? And we’re all “stealing” along with them, aren’t we? It would be the same as someone who steals a case of beer and the rest of us drink it.

So it does not make a difference if Jones/Mann and co *believed* the needed to fight back against those horrible sceptics. The fact is they engaged in unethical activity that corrupted the peer review process when the blackmailed a journals and editors for publishing papers they did not like.

There are always extenuating circumstances considered in a court of law and self-defense is always considered when determining guilt. I think they believed that the peer review process was being corrupted by the skeptics and the politicians who were on their side. The journals and editors realized that the papers they published were insufficiently vetted and were very inadequate. If scientists feel a journal has been corrupted by political motives of the editors, I can see how they would refuse to publish there.

Greenaway (Comment#24993)
November 26th, 2009 at 9:48 pm

Like people hacking CRU’s server and stealing emails and documents? And we’re all “stealing” along with them, aren’t we? It would be the same as someone who steals a case of beer and the rest of us drink it.

So how drunk did you get on the Pentagon Papers, or are you too young to remember?

Raven (Comment#24990)
What’s your evidence that they blackmailed journals and editors? I’ve seen indications that they intended to complain about papers they believed to be substandard, which anyone can do (and lots do), and that they may have threatened not to send their papers, which again is their choice. Scientists usually choose to publish in journals that they think are publishing good papers.

I wasn’t of drinking age at the time, but since I’ve read all the emails, I am as guilty as the next. I was just making a point in response to a comment about people doing unethical things and thinking they are behaving ethically. I’m sure that skeptics believe that stealing email and code and then consuming stolen emails and code is ethically justified just as I am sure that the scientists in question did what they did because they felt they were ethically justified.

Greenaway (Comment#24996)
November 26th, 2009 at 10:05 pm

I’m sure that skeptics believe that stealing email and code and then consuming stolen emails and code is ethically justified just as I am sure that the scientists in question did what they did because they felt they were ethically justified.

We’ll have to see if the e-mails were “stolen” or if they were “leaked” by a whistleblower.

I work in private industry, and we are constantly told that our e-mail is not private. I believe most government organizations tell their employees the same thing.

As far as the FOIA ramifications, this will all be decided by what’s “legal”, not by what’s “ethical”.

Glad it’s not me.

Greenaway–There are plenty of warmers and coolers who think stealing the emails was wrong, but reading them is fine. Lots of people here have gone on record to say the police should investigate to discover whether someone hacked into the CRU servers and/or the RC server (or any servers), but we are discussing the emails. For my part, I think it’s perfectly fine to discuss the emails, had the discussion why I think it’s fine wayyyyy wayyyy back in comments. (You will note that many people must think it’s fine to discuss the emails. After all, gavin, Andy Revkin, George Monbiot are discussing them. )

JohnM

I work in private industry, and we are constantly told that our e-mail is not private. I believe most government organizations tell their employees the same thing.

Same at national labs. I don’t know what the situation is at CRU but I think work emails can be requested under FOI. Honest to goodness personal emails are sent using honest to goodness personal email accounts; these cannot be obtained through FOI. (I understand these can be obtained for free from such services as Gmail/ yahoo/ and sometimes ones ISP.)

If you work at a national lab and don’t want your private emails to end up subject to FOI, you use your personal account for private emails and use the work one for work.

The CRU zip files contain work emails that appear to have been either leaked or stolen. So, on the one hand, they are not really private but on the other hand they weren’t obtained in an entirely orderly way.

That said– how they were obtained does not alter the meaning of the contents. The contents reflect badly on a number of climate scientists.

@ Simon Evans (Comment#24910)

Reads as a credible paper, but how do I know that the principles involved aren’t pushing an agenda? After all, it’s pretty clear that climate science has been corrupted based on the Climategate files. How am I to know whether these are honest scientists or simply more of the same who haven’t yet had their e-mail either hacked or outed by an insider (depends on whether you take Gavin as a credible source, I suppose)?

There are three points in all of this:

1) It’s very easy to cast aspersions on an individual or a paper based on nothing more than anecdotes
2) That practice has been made much easier and much more credible as the result of undeniable evidence that SOME leading lights in the climate science community have conspired to obstruct FOIA requests and have clearly made requests to their peers to delete e-mails that are undeniably the subject of FOIA requests
3) Folks like you, Simon, that refuse to condemn such behavior make the work of your arch enemies, the Denailists ™ much easier.

In a classic cognitive therapy setting, the behavior in point 3) above would be classified as dysfunctional. A more pop culture oriented way to put it would be Dr. Phil saying: “How’s that workin’ for you?”.

So, we come to the question I put to you in another thread that you used the “false dilemma” argument to avoid answering. Is the behavior described in point 2) above:

1) A good thing
2) A bad thing.

We’ll have your answer now Simon. Or perhaps not, based on previous responses to the same question. Please don’t divert to the data. This is about ethics, principle and law.

BTW, to be clear, I have no reason to believe that the author and co-authors of this paper are anything but principled scientists, but of course I have no way to KNOW that that is the case. Should everyone involved in climate science be painted with the same brush? Of course not, but that’s what the Denailists ™ will do, and Simon apparently feels that that’s 1) A good thing

OA

Nick Stokes (Comment#24995)
It does not make a difference what their opinion is of the papers since it is just their opinion. The cannot argue that a journal that accepts papers they do not like is biased. If they do not like the papers they can ignored them, submit a comment or submit their own paper. The could also choose to not submit papers to the journal in the future.

What is unethical is to threaten a journal to refuse to cite or submit papers to a journal unless the editors stop accepting papers that do not meet with their approval. That is blackmail – clear and simple.

It is also unethical to seek to get an editor fired because the editor might be sceptic. In this case, they succeeded in getting the editor removed.

There is no room for interpret or justifications for what they did. They had ethical choices and they choose not to use them.

I don’t think people should have to, by default, post their code for anything not appearing in a journal article. Posting code has little to do with reproducibility. If the calculation and its description are clear enough, write the code yourself. If you run someone’s turnkey code and you get the same results, well, shocker. If Ben Santer published all of his code for Santer et al. 2008, and I compiled it and double-clicked the resulting executable and found the same results, that doesn’t mean anything. I replicated Santer’s TLT/TMT data by implementing the same algorithms and data that he claimed he used and found he implemented them correctly. Was it easy to hunt down all the information that I needed? No. Did I go down plenty of dead ends? Yes. Was it easy to efficiently code the problem? No. But sometimes if you want to check someone’s work, you need to do some heavy lifting of your own. Would I post my code? Probably not because who’s going to download almost 150 GB of data to see if I’m wrong or right? Sometimes code is so complex that looking at it only gives you a decent idea of how it works. If you don’t have data to feed it, you’ll drive yourself nuts trying to see what the output would look like.

It’s worth noting that people who write code often have their own style and problem-solving methodology and that can sometimes be a hindrance to other people understanding the code. Many times I’ve looked at SteveM’s code and either thought “What the hell is he doing?” or “why is he doing it that way?”

Chad,

so, what do you do when you try to implement the procedure/algorithm and it doesn’t give the same result??

you can spend much time going down the wrong road trying to figure out what you did wrong when the author did not correctly implement it, forgot he made a change in it, did something completely different…

having running code along with the algorithm is also an excellent check for the author before he submits the turkey in the first place!!

Basically you and Lucia are missing the extra care the author would typically put into the work if they must meet the requirements of releasing a decent paper with the procedures/algorithms included, data, code, and anything else needed to duplicate their work.

The other issue is, do you really believe Mann, Briffa, and the others would have released their Hockey Sticks if they were REQUIRED to provide all the information? If they had, how long would it have been before Steve, you, or others had debunked them?

Raven, #24995, this is just a different viewpoint. You say they are motivated by an objection to to sceptic papers; they would say, genuinely I believe, that their objection is to bad papers. Now it may be that they judge the papers unduly harshly because of their sceptic slant, but there’s no way we can resolve that. The fact is that if they believe a journal is publishing bad papers, they can decline to send their work (and tell the editor why). And they can, in appropriate ways, seek to have the editor changed.

I imagine you’re referring to this Mann fragment:
Proving bad behavior here is very difficult. If you think that Saiers is in the greenhouse skeptics camp, then, if we can find documentary evidence of this, we could go through official AGU channels to get him ousted. Even this would be difficult.
It’s not well expressed, but basically correct. If you believe an editor is admitting bad papers through some factional allegiance, then you can try to go through official channels to get him ousted. That’s what the channels are for. GRL is published by AGU, and the members have a say. And as he says, it would be difficult. You have to make a strong case and convince a lot of people.

I saw nothing in the emails to say that anyone did that, although I believe Saiers did later lose his position.

kuhnkat,
What do I do? I keep at it. When I first attempted to replicate Santer, I was getting temperature data that was overall the same, but had too much variability. It was because I was using the RSS supplied weighting function and not the global-mean static weighting function that Santer used. It would have been nice for Santer to publish his source code. It would have simplified matters a bit, but it wasn’t crucial.

For the record, I recently finished up a climate-related programming task for a well-known researcher and the results of which might get published. I wrote and documented the code in a manner appropriate for publication. I also wrote a brief rundown how the program overall operates and some specifics that went into the calculations. But as far as my blog goes, I don’t publish code because I don’t see the need for it. Also, I’ve accumulated a large number of functions which greatly simplify a lot of my work by hiding the complex stuff behind function wrappers. I’m considering releasing it as an R package.

As far as your question about Mann/Briffa/others, I don’t understand what that has to do with this. SteveM has done plenty of work on Mann/Briffa without much source code.

… is also an excellent check for the author before he submits the turkey in the first place!!

Still have Thanksgiving on your mind? Hehe.

“I’m sure that skeptics believe that stealing email and code and then consuming stolen emails and code is ethically justified just as I am sure that the scientists in question did what they did because they felt they were ethically justified.”

In regards to the FOI, they consulted with the FOI people themselves. When they put their case that the requests were supported, they were supported by the FOI people. Nothing unethical about that, they complied with the act.

OTOH, what can be done with raw data.

http://wattsupwiththat.com/200.....icial-one/

He has published a lie. The reasons for the adjustment have been public for years, as has the data. Yet this is one more meme that has escaped into the denialosphere with no ’sceptical’ inquiry at all.

Bugs, I saw that to. NIWA has posted a response. Here’s a more in-depth look at the allegations.

I’ve seen and read a few interviews of von Storch in the German media. The poor professor is in a very hostile environment here. Germany has practically become the Vatican of the AGW Religion, dissenters can be burned at the stake here.
In his interviews he always appeared to to be sitting on the fence, not wanting to take either side. I surmise he didn’t want to join the religion because he knew it was all bogus. At the same time, he didn’t want to join the sceptic side for fear of facing the brutal wrath of German Green politics and media.
He really ought to come out and say more now. He ought to sign the open letter at EIKE and give them an interview. That is, if he still has any love and respect for the field of science. Perhaps an interview with DIE WELT newspaper.

@bugs.

You said:

“In regards to the FOI, they consulted with the FOI people themselves. When they put their case that the requests were supported, they were supported by the FOI people. Nothing unethical about that, they complied with the act.”

Have you read the Climategate e-mails? Do you know what the definition of “is” is?

Plug fingers in ears, close eyes tightly and sing LA, LA, LA as loudly as you can. Rinse and repeat.

Sheesh!

OA

Nick Stokes (Comment#25005)
I saying it does not make a difference what their motivation was. I am saying it unethical no matter what the motivation.
They simply have no right to second guess the editors judgement.
By doing so they compromised the process and we can longer trust that sceptical papers are getting a fair chance to be published.

OzzieAardvark (Comment#25000)

… evidence that SOME leading lights in the climate science community have conspired to obstruct FOIA requests and have clearly made requests to their peers to delete e-mails that are undeniably the subject of FOIA requests
3) Folks like you, Simon, that refuse to condemn such behavior….

I have repeatedly, and from the first reading of the relevant emails, stated taht I think the FOI matters are a concern and that they should be investigated.

I am fully prepared to ‘condemn’ behaviour on an if – then basis. So, for example, if data has been destroyed to prevent its release under FOI then I condemn that. What I am not prepared to do is to condemn individuals on the basis of pre-selected emails without knowing the full context and without knowing their response to concerns expressed (all of which would be explored in an investigation). That is about “ethics, principle and law” – I don’t recognise your versions of those concepts.

Lucia
I just popped in to see what you all think about CRU etc.
This database Harry thing reminds me of the guy hired by John Cage to get the great mans ‘music’ down on paper, of whom it was said he could ‘orchestrate the raisins in a fruitcake’.
Spose I’m just an old cynic…

Simon

What I am not prepared to do is to condemn individuals on the basis of pre-selected emails without knowing the full context and without knowing their response to concerns expressed (all of which would be explored in an investigation). That is about “ethics, principle and law” – I don’t recognise your versions of those concepts.

This is why I think an external investigation by government officials who are not climate scientists is required. We need more information to establish precisely what was going on. This information will also help those involved in the peer review process decide how it can be structured to be more transparent and less likely to be influenced by any small groups of activists.

P Gosselin–
Sorry you got moderated.

There is at least one other possibility. He is not a skeptic, but is appalled at the behavior of the activist crowd. It’s one thing to believe the preponderance of the evidence points to warming– even alarming warming. It’s another to behave like a little cabal and work to stifle dissent or just less alarming views.

Maybe a song will help some people deal with this uncomfortable and unprecedented turn of events. You guys remember ‘Drive’ by The Cars, don’t you? Derided by many as horrible corporate pop, it’s still a great song.

♫ Who’s gonna tell you when
It’s too late
Who’s gonna tell you things
Aren’t so great

You can’t go on
Thinking nothing’s wrong… ♫

FYI, Benjamin Orr did the vocals on this one. He left us too soon. RIP.

Andrew

One of the important revelations in the LEAKED emails (actually a confirmation of Wegman’s conclusions) is that “peer review” became “buddy review,” wherein the Team groupies writing the papers were also reviewing them. I have a hard time believing that the editors of the journals were not aware of this problem. IMHO, this is a BIG black eye for the sanctimonius folks that think that there is no scientific truth outside of the peer-reviewed process.

Climategate e-mails sweep America

“At this rate, Copenhagen is going to turn into a comedy convention with the real world laughing at these liars.”

http://blogs.telegraph.co.uk/n.....rade-laws/

Andrew

Zorita has called for banning Jones, Mann and Rahmstorf from the IPCC

http://noconsensus.wordpress.com/2009/11/27/357/

For those who don’t know he’s published with VonStorch previously and you can also find his response at Lucia’s link in the head post.

bugs:

He has published a lie. The reasons for the adjustment have been public for years, as has the data. Yet this is one more meme that has escaped into the denialosphere with no ’sceptical’ inquiry at all.

Are you telling us that averaging the unadjusted temperatures doesn’t give a flattened curve? To be able to make this claim, you certainly must have checked yourself.

We’ll be waiting for the posted figure from you showing that the flattened curve “is a lie”.

I’m starting to think that your definition of “lie” is anything that calls into question your religiously held beliefs.

[...] on Von Storch Cru Reactionbugs: Are you telling us that averaging the unadju… »Alexander Harvey on Applause for Judy [...]

Simon,
You are in the running for gold medals for mental gymnastics in the ‘rationalization’ division.
In my work, from time to time I get questioned rather directly on what I did vs. what a client says happened.
At that point, I simply turn over everything I did regarding that issue, good bad or indifferent. I answer questions promptly and directly. the truth has a great way of coming out.
The response of those outted in the leak is remarkably different.
The stench of fraud is strong in these leaked documents, and they are clearly only a whiff based upon a small release. History advises us the rest will likely be much worse.
The theory of global warming that involves dangerous, apocalyptic changes due to CO2 in the atmosphere at this point has no trustworthy proof.

I’m trying to decide if bugs is better off for having been thoroughly brainwashed by Jim Hansen rather than Jim Jones.

When the heat and fury have died down two important questions will remain.
1. How do you manage the peer review process to ensure a reasonable standard of publication and still allow contradictory papers to be published? After all few papers can have had as many problems as Mann’s Hockey Stick, savaged by the Wegman report, or his more recent paper with upside down sediments.
2. How do you ensure reproducability in the digital age? In the past scientists would describe their experiments in great detail so others could repeat them and confirm or question their results. For example now someone may write a paper using commercial subroutines; should all testers be expected to buy this software. Computer code can have commercial value; how can you prevent scrutineers from ’stealing’ this code. To prepare a data base of temperature or preciptiation takes time and money; should this data base be handed free of charge to anyone who asks?

There are no easy answers.

Carrick (Comment#25036)

bugs:
Are you telling us that averaging the unadjusted temperatures doesn’t give a flattened curve?

NIWA has detailed the case of Wellington. Yes, unadjusted data is flatter. The reason for the major adjustment is that in 1928 the measurement site moved from Thorndon (alt 6 m) to Kelburn (alt 128 m). The effects of altitude on temperature are well-known and easily quantified. NIWA has a long overlapping record from the airport, nearby and near sea level, to make that quantification.

The NZCC just grafted the Thorndon and Kelburn data without adjustment. This is not progress. Yet WUWT wants us to believe this is Climategate II.

The adjustments for Wellington are explained here.

Chad,

“Still have Thanksgiving on your mind? Hehe.”

Yup, especially the basting part!!

Does anyone contest that this is where the NIWA station in question is currently located?

http://wattsupwiththat.files.w.....tation.jpg

Hey, Nick. Are you REALLY still trying to defend the undefensible? What do you think about this post by Mc:

http://camirror.wordpress.com/.....-deletion/

??

What do you think about the leaked (definitely not “hacked”) emails? And the particularly disturbing comments in the computer code?

John M (Comment#25078)
Does anyone contest that this is where the NIWA station in question is currently located?
Yes. According to the cited documentation, this is the NIWA building in Khyber Pass, in Auckland. Nowhere near Kelburn, Wellington. It sure doesn’t look like Kelburn.

jae (Comment#25084)
Hey, Nick. Are you REALLY still trying to defend the undefensible?
No, I’m trying to pin down the intangible. As soon as I make progress clarifying one allegation, the subject switches to another, as here.

An example, on those code comments that you mention, is #24892.

Steve’s post rests on the Guardian quoting Phil Jones saying “We’ve not deleted any emails or data here at CRU. ” There has to be some context there; it can’t be literally true. I presume the context was “subject to an FOIA request”.

And on that, we don’t know yet. Just as you have know knowledge of whether the emails were leaked or hacked.

Nicko:

“And on that, we don’t know yet. Just as you have know knowledge of whether the emails were leaked or hacked.”

Oh, come on Nick, I know that you know better. You appear to be getting very desparate, as are your friends. As explained by MANY posters over the past few days, there is virtually NO WAY that a mere “hacker” could get into the data base and select precisely those emails that would be affected by a FOIA request. It is clear that a whistleblower got sick of the disgusting SHIT that has been passed by some famous creeps as “climate science.” So give this “hacker” crap up, before you look even dumber.

Make no mistake, you and your spin-dokter ilk may prevail on the political court, for the near time, but y’all have LOST, BIG TIME, on the scientific court. IMHO, you are tacitly supporting the most egregious, disgusting example of fraud and false science in all of history. You will lose in the science annals and will help to make the public more wary of scientists for a long time. And your failure to compromise on even the most minute point makes you look even more pathetic.

Thanks Nick.

John M #25090
On further reading the documentation, I see that it isn’t even a met station, but an air quality monitoring station. That’s WUWT!

Nick Stokes,

“Steve’s post rests on the Guardian quoting Phil Jones saying “We’ve not deleted any emails or data here at CRU. ” There has to be some context there; it can’t be literally true. I presume the context was “subject to an FOIA request”. ”

Actually Nicky, it is WORSE THAN YOU THINK!!!!!

In the early e-mails they were discussing how to get around FOI even before they received their first request. That tends to invalidate your apology.

Now, read that quote again.

“We’ve not deleted any emails or data here at CRU. ”

What about that data Nicky?? Phil Jones actually said he might get rid of the data before he would release it. Weren’t we just told that due to storage issues they got rid of all the original temp data and it is gone forever?? Are you going to nit pick that throwing away data is not the same as deleting data?? (that is providing it WAS never digitized) Does the original temp data that would be required to do a full audit of their temp product not count as anything??

Does the fact he said he would get rid of it years ago and now says it is gone not suggest anything to you??

Sorry Nicky old boy, go ahead and parse it all you want, Phil Jones is a LIAR. He needs to shut up, lawyer up, and settle on a story instead of making a laughing stock of the whole community!!
Mann and a couple others should do the same.

KK
Again my complaint about changing the subject, which was whether CRU deleted info that was subject to a FOI request. Nothing you’ve said bears on that.

Yes, of course PJ does not like FOI requests, and will seek to do the minimum that complies. That’s what he was talking about in the earlier emails, and he is actually quite entitled to seek to do that.

Nick Stokes:

The problem is that (1) Phil Jones emailed his buddies that he would destroy the raw data before turning it over. The unmistakable inference is that the data existed at that time and that it existed under his personal control.

And (2) he later told FOI requesters that the data did not exist.

Did he lie to his friends? Lie to the FOI requesters? Was any data in fact destroyed? If so, when?

I do not expect him to offer a detailed public explanation of this contradiction because by now he should have replaced Andy Revkin’s number in his speed dial array and replaced it with that of his solicitor. For his sake, I hope so.

Whether or when anything was deleted, who owns that data, whether the use or maintenance of that data was material to funding requests, whether communicating an intention to violate legal duties constitutes conspiracy, whether advising others to delete emails after communicating an unlawful intent is a form of obstruction…. an investigator who really wanted to pick the flysh*t out of the pepper could probably find a lot in the emails alone.

NOTE: I do NOT accuse Dr. Jones of anything. I merely note that given the content of the documents that have come to light, this is the kind of thing a lawyer would immediately worry about for a long, billable period of time. The notion that is all nothing but a few snide remarks about denialists is itself a form of denial.

My guess is that some careers will take a hit and that institutions will move to reduce liability and to protect future funding while like-minded politicians provide cover wherever possible. But you never know. These kinds of things sometimes take on a life of their own, especially if somebody says something stupid at the wrong moment…

A few observations on the issue at hand:

People assume the following:

1. That the emails were “leaked” vs. “hacked”, however, there is no evidence to conclude firmly either way. They could be leaked or hacked. If leaked, the leaker could be in trouble, depending on the laws of the land regarding whistle-blowers. If hacked, same thing applies. Even if these materials are subject to FOI requests under the law, stealing them is not allowed. There are proper channels to go through to get access to these kind of documents and there are legitimate reasons to deny requests for information. People who downplay the issue of the emails being stolen are being hypocrites — if it’s wrong to destroy emails, it must certainly be wrong to steal them, regardless of the motive.

2. That no further context is needed to understand the meaning and intent of the authors of the emails. In other words, reading the email in isolation is good enough; that an email isolated from the emails that preceded it and followed is sufficient to grasp its meaning and significance. Now, I admit that some of the emails appear to be pretty damn incriminating — but what if we had access to the whole lot and a few emails down the line, one of the writers took back what they said, said they were just joking of course, and we were able to see that it was bluster, frustration, or a joke, etc. What if Harry is an idiot and just didn’t know programming well enough to reproduce the results? As unlikely as that may be, it is still a possibility. We don’t know who Harry is or what his skill level is, etc. I say that unless we have all the emails and all the code and know all the principles involved and have their side of it, we can’t conclude with certainty what took place. Sure, people can speculate and conjecture, but that is it.

3. People are assuming that all these documents are pristine, delivered to us exactly as they were on the server at the CRU. But, since they are a subset of many more documents, it is possible — probable — that these particular emails were cherry picked to make a point. People making points very often leave out evidence that doesn’t support their point or contradicts them.

It is entirely possible that some of the documents have been “doctored”. We know that some of them are bona fide because the principles involved have commented to try to explain themselves. However, I don’t know about you, but if I ever saw a file like the Harry file from the data techs where I work, I’d have a word with them and tell them to be more professional. All those comments would be seen as completely unacceptable in my organization. I can’t imagine making comments like that on a paper or document a supervisor or lead researcher produced. It may be that Harry really did make all those comments as he worked on the file, but it may also be that those were doctored to make the work appear worse than it is.

This is obviously one of the most politically charged issues in our time. Because of that, because of the stakes, because of the money and reputations and careers that are to be made and destroyed, not to mention the possible consequences to the globe, this is ripe for hoaxes and frauds — on either side.

I think the jury is out on the meaning of climategate and until we have more evidence one way or the other, I’ll keep an open mind when considering the emails and what conclusions should be drawn.

For my part, I think it’s perfectly fine to discuss the emails, had the discussion why I think it’s fine wayyyyy wayyyy back in comments. (You will note that many people must think it’s fine to discuss the emails. After all, gavin, Andy Revkin, George Monbiot are discussing them. )

Obviously, I’m participating in the discussion of the emails and have read them. That doesn’t negate the fact that they were likely obtained illegally and by reading them, we are all taking part in an illegal act just as surely as if we drank the stolen beer. We each make a choice based on our own ethics. It’s like watching a pirated video — maybe you didn’t buy it, maybe you didn’t pirate it, but you are consuming it all the same and benefitting from an illegal act. For the record, I don’t buy or watch pirated videos. The reason I’m reading these emails is so I can, for the purposes of research, gauge how both sides respond. It still feels quite uncomfortable.

greenaway:

That doesn’t negate the fact that they were likely obtained illegally and by reading them, we are all taking part in an illegal act just as surely as if we drank the stolen beer

If the emails contained protected trade secrets that would be true. It’s not even close to truth here. These emails are exactly the opposite—they are subject to public disclosure.

My suggestion: Stop lecturing people about things you don’t know or understand.

Carrick (Comment#25101)
My suggestion: Stop lecturing people about things you don’t know or understand.
No, that would be my advice to you. The emails are not subject to public disclosure. They are subject to the FOI process, in which all sorts of protections exist for information that should not be disclosed. This is all violated by this leak/hack.

For example, 1140554230.txt contains confidential editor discussions about publishing one of Mann’s papers, including review matters. Now I’m sure the journal would have objected strenuously, and successfully, to that being published. Scientific journals just can’t operate unless they can get frank confidential opinions from referees. And of course having those confidential opinions revealed is a gross breach of faith.

If the emails contained protected trade secrets that would be true. It’s not even close to truth here. These emails are exactly the opposite—they are subject to public disclosure.
My suggestion: Stop lecturing people about things you don’t know or understand.

Emails may or may not be subject to public disclosure — that’s up for the FOI officials who do the analysis to determine. Until that time, any attempt by unauthorized people to gain access to the server and take materials contained therein are illegal.

I work for a very large org that is subject to FOI requests frequently because of our information and work. We have computer techs who work full-time to check our system to ensure we are not easily hacked. We have numerous security systems in place to keep unauthorized people out. If anyone did hack in and take materials, you can be sure there would be police involved. It doesn’t matter that the same person could submit an FOI to get the information. Theft is theft. Besides, there are costs involved in retrieving information for FOI requests and that is passed on to the person requesting the information.

Do you actually work for an organization with a computer system or are you just making this up as you go?

“The problem is that (1) Phil Jones emailed his buddies that he would destroy the raw data before turning it over. The unmistakable inference is that the data existed at that time and that it existed under his personal control. ”

He was angry about what he felt were vexatious requests using FOI done in bad faith, that is, not for genuine scientific enquiry, but merely for the purpose of vandalising science. He complied with those responsible for administering FOI and put his put of view to them. Those administering the FOI system agreed with his evidence that the requests were indeed vexatious. A quick look at ClimateAudit was apparently quite convincing, unsurprisingly.

Nick:

They are subject to the FOI process, in which all sorts of protections exist for information that should not be disclosed. This is all violated by this leak/hack.

It’s got nothing to do with the FOI.

In the US, any email I write as an official part of my work (as a government employee), is not considered “private”. That is to say they can be released without my waiver or authorization, and it doesn’t require an FOI or my consent for my employer to release them.

And once released, regardless of how they were released, they are not protected private property regardless of how they were released.

The only exception to this I know of is, as I mentioned, trade secrets and the like. If you receive information of that sort that was illegally obtained, it is considered stolen property and you are obligated to destroy it, or face criminal penalty.

Keep crying the crocodile tears both of you. With practice, you’ll get good at it and maybe convince people you mean it.

greenaway:

Until that time, any attempt by unauthorized people to gain access to the server and take materials contained therein are illegal.

This is partially true but has nothing to do with my point, which was about whether they are considered “stolen property” once released. I’ll let you reread what I said rather than regurgitate it in small words for you.

It is “partially true” because applicable releases of information by whistleblowers are protected under both US and British law. I’m guessing if this was done by an employee of UEA, it would be protected.

Finally, if the information was left in an open directory on the server, it would also not be protected or illegally garnered.

Again, my advice…stop lecturing people about things you know not.

bugs is showing his psychic abilities. He purports to know what Phil Jones was thinking.

I should have said ” it doesn’t require an FOI or my consent or a lawsuit for my employer to release them.”]

They can also be released under a court order.

“bugs is showing his psychic abilities. He purports to know what Phil Jones was thinking.”

No, read the email where he talks about FOI agreeing that the requests are vexatious and in bad faith. A tour of sites like climateaudit was apparently quite convincing.

By the way Nick, do you know what “subject to public disclosure” even means? You denied they were, then admitted they could be released by a FOI request.

bugs: “No, read the email where he talks about FOI agreeing that the requests are vexatious.”

OK, I’ll bite.

In which email did he use the word “vexatious”?

(ANS:: None.)

I used the word vexatious because it describes the situation, I did not quote the letter.

bugs:

I used the word vexatious because it describes the situation, I did not quote the letter.

Which makes it your interpretation of events, doesn’t it?

You do that a lot.

The FOI agreed with their request to not have to respond to the FOI requests. Do you have a better reason?

bugs:

The FOI agreed with their request to not have to respond to the FOI requests. Do you have a better reason?

I believe Phil Jones explained in one of his emails that the FOI agent would never agree to release any documents to McIntyre.

Do you agree that is an accurate paraphrase?

Nick & Greenaway… I should mention that my comments relate to emails that are sent or received through my employers email server. It gets a bit more dicey if they are sent from and to a private server (see Bush email scandal).

And for the record, I’ve had emails released without my giving explicit permission in the past. Fortunately I didn’t say anything embarrassing in them. As a public employee one should always assume somebody is reading over your shoulder when you are writing emails.

Jones referring to requests for data.

http://www.eastangliaemails.co.....346299.txt

“> At 14:54 20/06/2007, you wrote:
>>Phil
>>It is nasty. It is also very inappropriate. Even were some problems to
>>emerge over time, those should be addressed in a new paper by these guys.
>>Unfortunately all they do is criticise.
>>Kevin”

Yep

bugs, that wasn’t exactly apropos. Are you always this random?

do you know how hard it is to find one of those emails again? It explains Jones state of mind, I believe.

“I believe Phil Jones explained in one of his emails that the FOI agent would never agree to release any documents to McIntyre.

Do you agree that is an accurate paraphrase?”

I think that was what happened.

meanwhile, i’m guessing mcintyre ignores evidence such as this

http://www.nature.com/ngeo/jou.....eo694.html

I’ll be waiting for a post from him to say that AGW is proceeding faster in areas than had been anticipated, and we should do something about it.

“Ben,
When the FOI requests began here, the FOI person said we had to abide
by the requests. It took a couple of half hour sessions – one at a screen, to convince
them otherwise
showing them what CA was all about. Once they became aware of the types of people we were
dealing with, everyone at UEA (in the registry and in the Environmental Sciences school
- the head of school and a few others) became very supportive. I’ve got to know the FOI
person quite well and the Chief Librarian – who deals with appeals. The VC is also
aware of what is going on – at least for one of the requests, but probably doesn’t know
the number we’re dealing with. We are in double figures.”

http://www.eastangliaemails.co.....330629.txt

bugs (Comment#25121)-Why would Steve take a formal position on policy? And why is this “evidence” so important in you mind anyway, that it alone merits something like that?

Besides, Steve has said plenty of times that he thinks there are justifiable reasons to “do something”. Don’t be such a pisser.

Wow. That’s damning. Do you have no clue what is wrong with that?

“Wow. That’s damning. Do you have no clue what is wrong with that?”

The FOI people support them? I know they see what’s wrong with climate audit and how it is not honest but vexatious.

Jones says quite clearly, he wants to take them on, using the scientific process he was trained in and supports. The cesspit that mcintrye has created does not attract him at all.

Perhaps this will explain part of the problem with making everything ‘open’.

“The thing is, climate scientists don’t expect better, and never have seen much better. The fact is that despite the endlessly inflating claims of billions of dollars going to climate science, the software engineering staff on the leading American climate model has been reduced from eight to six, with the cuts coming from user support and documentation. This is why CCSM, though on release 3.1, is only documented to 3.0, and that only badly.

Yes, it’s true, it’s a trillion dollar problem, but the annual budget for studying it (once you subtract all the impact studies and the ecological observation programs and the satellite launches and satellite base stations, all very valuable but not what most people think of as climate science, and get right down to what is actually climate science) is about that of a high-profile Hollywood movie, and that for model development and maintenance on each individual model is far less. From 2003-2008 NCAR had to lay off approximately 55 people and lost another 77 positions due to attrition, totaling roughly 16% of NCAR positions, because of sub-inflationary NSF funding and decreases in other agency support. (Statement of program reductions at NCAR).”

http://initforthegold.blogspot.....gripe.html

Openness costs money.

bugs has a million excuses. Any reason at all is good enough for him.

Too bad he’s in a minority.

I’ll be waiting for a post from him to say that AGW is proceeding faster in areas than had been anticipated, and we should do something about it.

Think there might be calibration issues? What the hell, let’s go spend 10 trillion dollars just in case there aren’t.

Bugs:

Openness costs money.

Jones has lots of money. You can stop shoveling the manure anytime you want.

Andrew #25124
I have no clue what is wrong with that. Would you like to spell it out?

Carrick
You’ve introduced the employer as a separate player. Yes, the employer controls the emails etc. But this misunderstands the privacy that greenaway and I are talking about. It isn’t what you need to engage in illicit relationships, communist subversion or whatever. It’s what you need to do your job properly. Write frank employee assessments and referee reports, grant evaluations, engage in all kinds of internal and external negotiations etc. In any rational employment context, the interest of employer and employee will coincide.

A proper FOI process will respect those needs. Hacking won’t, and that is very damaging. People stick their necks out to do their job, and get stomped on.

“Jones has lots of money. You can stop shoveling the manure anytime you want.”

Jones doesn’t have lots of money. Read the in it for the gold link. Jones has a budget, and limited resources to allocate to tasks.

Nick:

You’ve introduced the employer as a separate player. Yes, the employer controls the emails etc.

Then we’ve actually agreed on something! Hallelujah!

Note that in addition to FOIs, legitimate whistle blowing is a protected form of information release both in the US and Britain.

The other main point is that email isn’t protected intellectual property unless it contains trade secrets. That is the part that I was responding to greenaway’s misunderstanding of the legal issues involved with electronic mail. Had the emails contained trade secrets, that would be grounds for being forced to delete the emails.

The reason that the control is via the employer is important is, had the emails come from a server where there was an expectation of privacy (corporate or state-owned servers don’t meet this requirement), then probably dissemination of the emails would be illegal, or at least open to a court challenge. Clearly that’s not the situation here.

As Overpeck said in one of the emails “Please write all emails as though they will be made public.” Good advise.

bugs:

Jones doesn’t have lots of money. Read the in it for the gold link. Jones has a budget, and limited resources to allocate to tasks.

I hope somebody’s sweeping up all that manure you’re shoveling bugs. It’d be a waste.

I have a far smaller budget than Jones and manage a much higher degree of transparency, having code and data archived (with very few gaps) back to 1984.

It doesn’t’ take that much time or money to do it right. Intuitively people know this, and when Jones refuses to do it, or you refuse to admit it, either way, you are further undermining public trust in the scientific process.

“It doesn’t’ take that much time or money to do it right. Intuitively people know this, and when Jones refuses to do it, or you refuse to admit it, either way, you are further undermining public trust in the scientific process.”

You have gaps :O.

I have worked with data and code as well, and it’s not easy to do it right, to have it fully indexed, commented, versioned, backed up, tested, documented, safeguarded, secure, (audited internally and externally), reviewed, debugged, maintained, supported and managed and planned.

Greenaway–
With regard to your claims people are assuming 1-4, you are mistaken. You will find that
1)

That the emails were “leaked” vs. “hacked”, however, there is no evidence to conclude firmly either way.

I and many people agree with you. Many here are mostly agnostic as to whether the emails were hacked or leaked. We think this should be investigated. If the release was illegal, then the police authorities should investigate and find out.
2)

That no further context is needed to understand the meaning and intent of the authors of the emails

Some think this on both the cooler and warmer side. However, you will find at least I and a number of others have indicated that further investigation by an outside party is required to find this out and advocate further investigations. (Do you advocate further investigations.)
3)

People are assuming that all these documents are pristine,

Oh? To the contrary. In comments, people have discussed whether the documents are real, fake, or a comment of both. I’ve said that I think the Harry file must be hacked because those comments are so outrageously damining it can’t be real.

4)

The reason I’m reading these emails is so I can, for the purposes of research, gauge how both sides respond. It still feels quite uncomfortable.

And what makes you think others aren’t reading for the same reason as you are? Or for equally ethical reasons? Are you the only one who gets your holy dispensation? I myself feel quite comfortable reading the emails.

You seem to operate on a huge number of assumptions of what people think. It might help if you actually read comments or ask people

hunter (Comment#25039) November 27th, 2009 at 10:34 am

Simon,
You are in the running for gold medals for mental gymnastics in the ‘rationalization’ division.

What? You think that saying there should be an investigation is a “rationalization”? I think it’s rational, that’s for sure.

In my work, from time to time I get questioned rather directly on what I did vs. what a client says happened.
At that point, I simply turn over everything I did regarding that issue, good bad or indifferent. I answer questions promptly and directly. the truth has a great way of coming out.

So there should be an investigation, don’t you think, by means of which people may be questioned directly?

The response of those outted in the leak is remarkably different.

We don’t know the formal responses to allegations that are being made. We would need an investigation to have such responses, wouldn’t we?

The stench of fraud is strong in these leaked documents, and they are clearly only a whiff based upon a small release.

You, and others, are making allegations of fraud. I don’t see evidence of fraud. Do you not think there should be an investigation to examine such allegations?

History advises us the rest will likely be much worse.

History advises us that investigations sometimes expose ‘much worse’ and sometimes exonerate those being accused. History also advises us that if you presume guilt without a proper process of investigation and the right to defence against allegations then you end up with lynch mobs.

The theory of global warming that involves dangerous, apocalyptic changes due to CO2 in the atmosphere at this point has no trustworthy proof.

What??? Because you suspect impropriety on the part of a few scientists you therefore reject all evidence of sea level rise, ocean heat content, satellite temperature analysis, ice cap mass loss, etc., etc.? Not to mention rejecting radiative physics theory? (There’s no ‘proof’ to be had anyway, the paleo reconstructions are certainly no such proof and the existence of the HadCRUT temperature anylysis isn’t either).

By describing what I have said as “mental gymnastics” are you suggesting that you don’t think there should be an investigation?

@bugs (Comment#25140)

bugs – other than a lack of discipline and motivation what are the difficulties involved in systematic and auditable working with data upon which global policy may be decided?

“There’s no ‘proof’ to be had anyway”

Watch out, Simon. Some Evilly-Intentioned Big Oil Bankrolled Polar Bear Hating Denier like me might interpret this as your admission that Global Warming is just another unfalsifiable hypothesis.

Andrew

bugs

I have worked with data and code as well, and it’s not easy to do it right, to have it fully indexed, commented, versioned, backed up, tested, documented, safeguarded, secure, (audited internally and externally), reviewed, debugged, maintained, supported and managed and planned.

This is why I think SteveMosher is going overboard to suggest this standard applies to blog posts However, this standard should be applied to the code for CRU and GISSTemp. These are production codes, supported by taxpayer funding, crank out numbers monthly and the results are used to guide policy.

Codes for journal articles fall somewhere between blog posts and CRU & GISSTemp. But the argument that SteveMoshers standard is too high for blog posts does not mean it’s too high for CRU codes used to create HadCrut etc.

How about this:

If someone is going to make a claim that includes EVERYONE’s involvement (i.e., something ‘Global’), then that someone has to be able to demonstrate the validity of the claim to EVERYONE, and answer EVERYONE’s questions and show EVERYONE the methods and the data.

If someone is going to claim something on his/her blog, then that person should follow the same pattern for the readers of the blog. It’s up to the individual reader to determine what to do about it, if anything, at that point. The more honest the presenation, the better an individual determination can be made.

Andrew

bugs:

You have gaps :O.

I don’t know for a fact that I do, but I suspect some data loss from a 1985 server crash.

I have worked with data and code as well, and it’s not easy to do it right, to have it fully indexed, commented, versioned, backed up, tested, documented, safeguarded, secure, (audited internally and externally), reviewed, debugged, maintained, supported and managed and planned.

Well that’s a lot more there than just having making data analysis turnkey. Data safeguarding is trivial of course.

“to do it right, to have it fully indexed, commented, versioned, backed up, tested, documented, safeguarded, secure, (audited internally and externally), reviewed, debugged, maintained, supported and managed and planned.”

Excuse me, this is what happens in many businesses and their resource partners everyday. My coworkers and I do it perpetually. I like to call it ‘doing my job’.

Andrew

If someone is going to make a claim that includes EVERYONE’s involvement (i.e., something ‘Global’), then that someone has to be able to demonstrate the validity of the claim to EVERYONE, and answer EVERYONE’s questions and show EVERYONE the methods and the data.

You don’t need to shout.

I’m afraid that it isn’t possible to demonstrate the validity of the claim to everyone. For one thing, we can’t all agree to what constitutes a valid claim. Validity is a subject of great scientific and public debate. Some people who are strongly committed against the whole notion of global warming will not accept any evidence.

Do we let them determine whether the rest act on a threat?

For me, it’s the preponderance of the evidence that ultimately convinces. While it is remotely possible that the vast majority of scientists doing climate research are hoaxers involved in the biggest scam of human existence, it’s unlikely. There may be a few less than ethical types as there are in any community, and some data that is rightly suspect, as there is in any discipline’s body of work, but the vast majority will be just like most people — they want to do the right thing and are honest.

To believe otherwise is to fall into a form of paranoia where one doubts everything.

Greenaway–

Do we let them determine whether the rest act on a threat?

We? In a democratic system, everyone gets a say. There are certainly a number of people who will not accept global warming based on any evidence. (I think I could name one or two.)

There really is no danger that these few people are going to overwhelm others and prevent action all by themselves. If the majority of people want action we’ll get action.

The stench surrounding the CRU zip file is going to make it more difficult to convince people in the center that those advocating the more drastic actions are communicating in good faith. Even if there is no smoking gun in those email, some key participants have given the impression that they are willing to spin information like tops, obstruct and generally paint a picture that may be somewhat truthful but is hardly “the whole truth and nothing but the truth.”

This is why I think SteveMosher is going overboard to suggest this standard applies to blog posts However, this standard should be applied to the code for CRU and GISSTemp. These are production codes, supported by taxpayer funding, crank out numbers monthly and the results are used to guide policy.

In my view, such demands are outlandish with a slightly thuggish aroma.

I agree that any research undertaken at the taxpayer’s expense should be of the highest quality and should be readily available to others qualified to review it for quality control purposes and validity, etc.

Academic research is subject to its own policies and procedures for validity and reproducibility. While certainly not perfect, as many frauds in the past have shown, it tends to work pretty well. Above all, institutions should follow their own policies — very often the laxness with which some enforce their own standards and policies is the cause of bad research and data. I think the Climate Research / Soon Baliunas fiasco and resignation of the editorial board proves this to be so.

I agree very strongly that good public policy is premised on sound science and research. Public policy also part of the political realm, and thus is subject to the vagaries of the political process and cycle. Politicians either come to the answer they are seeking or they do not act, for they won’t usually act against their own interest (which is to get reelected.) Policy decisions are political decisions, ultimately. In principle, one wants the policy decisions to be premised on the best science available, but assuming that is what actually happens in real life is, shall we say, naive.

That doesn’t mean we can’t demand that it is, but having worked in private industry, the public sector and academia, I tend to be realist about these things rather than quixotic.

We? In a democratic system, everyone gets a say. There are certainly a number of people who will not accept global warming based on any evidence. (I think I could name one or two.)
There really is no danger that these few people are going to overwhelm others and prevent action all by themselves. If the majority of people want action we’ll get action.

Of course, I’m referring to the “we” of the majority.

I would like to agree that I need not fear the hard-core deniers but I am not so sure. There are very strong interests who will use the most vocal deniers to cast enough doubt over the whole to delay it or even prevent any real action from taking place. There are politicians who will impede action because of their benefactors. The truth is this — global warming — could be 100% real, 100% valid and its predictions could be 100% accurate and it is entirely possible that the political process could lead to inadequate or no action.

The stench surrounding the CRU zip file is going to make it more difficult to convince people in the center that those advocating the more drastic actions are communicating in good faith.

Of course, and that is a possible motive of those who hacked the CRU. I know this sounds elitist, and if so, so be it, but the average person is not equipped — in my view — to be able to sort through the debates and evidence and judge what is valid and what is not. They rely for their opinions on the media, their friends, and their personal biases and ethical systems. Of course I believe in democracy, but it is not a guarantor of good public policy. There is no better system but it’s not perfect.

Even if there is no smoking gun in those email, some key participants have given the impression that they are willing to spin information like tops, obstruct and generally paint a picture that may be somewhat truthful but is hardly “the whole truth and nothing but the truth.”

I acknowledge that some scientists, based on their own research, have become advocates for one position or another on global warming. They are citizens as well and have every right as such to take part in the political process. I know several who have pretty much written off the next century or ten, believing that we have already passed critical points and are in for a wholly new climate no matter what we do at this point in terms of greenhouse gas emissions. The only solutions remaining are extreme geoengineering projects that in themselves carry unanticipated consequences and risks.

That these scientists don’t march on Washington based on their judgement of the research is a mystery to me. Frankly, given how politicized this branch of science has become, I think they’ve just been burnt out.

“I’m afraid that it isn’t possible to demonstrate the validity of the claim to everyone.”

That’s a problem, isn’t it. Perhaps someone could start by demonstrating the validity of it to anyone/just one person, and see how it goes.

Andrew

That’s a problem, isn’t it. Perhaps someone could start by demonstrating the validity of it to anyone/just one person, and see how it goes.

So, you feel confident that you have read it all and considered it all, and none of it, not one whit, has been valid.

Next.

“So, you feel confident that you have read it all and considered it all, and none of it, not one whit, has been valid.

Next.”

Greenaway,

I didn’t say that at all. I said maybe a demonstration to one person was a good start.

Maybe you could describe how Global Warming was demonstrated to you and highlight for us the most convincing evidence you saw.

Andrew

Andrew,

Do you mean a demonstration that the world has warmed, or that GHGs will be warming forcings, or that continued increase of GHGs will pose risks of higher temperatures, or that there are probabilities of positive feedbacks, or what? Your question’s a bit vague at the moment. Naturally, a full answer to everything would run to thousands of pages, so perhaps you could ask more particular questions.

Simon,

Yes. Let’s start at the beginning. How was “the world is warmer than it was” demonstrated to you? What happened that caused you to think that, when you didn’t before? Or have you always believed that and cannot recall ever thinking otherwise?

Andrew

“Oh? To the contrary. In comments, people have discussed whether the documents are real, fake, or a comment of both. I’ve said that I think the Harry file must be hacked because those comments are so outrageously damining it can’t be real. ”

Harry is relating the problem of data collected over a century, back when it was not considered important that they be more systematically cared for, nor what they could be used for in the future. It is damning of no-one, not the people in the present who have to deal with the data problems, nor those in the past who made the collections and stored them. Realclimate has more to say on this.

I think Greenaway and Andre_KY need to meet on the banks of Weehawken sometimes soon… Anyone want to act as their seconds?

In a reply regards Jones’ threat to delete data rather than hand it over, bugs opined in #25104:
.
“He was angry about what he felt were vexatious requests using FOI”
.
Jones wrote:
“If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone.”
.
Precognitive anger regarding FOI requests he was yet to receive?

lucia (Comment#25183)
November 28th, 2009 at 3:10 pm

I think Greenaway and Andre_KY need to meet on the banks of Weehawken sometimes soon… Anyone want to act as their seconds?

If I were to act as a second for either one of them, I’d need a fifth to get me through it (or at least a pint).

Yes, call me Andre_KY, from now on. It gives me an air of European Sophistication, although then you have the “KY”, which is the opposite of that.

Andre

In light of the Truly Surprising ClimateGate Scandal I found a new AGW theme:

♫ The ship is going down fast
But I won’t be around
I’m jumping in a lifeboat
And heading for dry ground

I’m not going down with the ship ♫

http://www.youtube.com/watch?v=qze06Ig2eQM

Andrew

Nick Stokes (Comment#25130)-Talking to FOIA people and persuading them-via pathos and ethos to make an exception because they think Steve is being a jerk? Oh heavens nothing wrong with that at all.

Both of you are clueless. There is no “jackass” except to FOIA.

Andrew, I live in the north where warming is having a considerable effect. I’ve been alive for almost five decades and have seen where I live clearly warm up since I was a child. The temperature record for where I live shows that my personal experience is correct and not just fancy. In fact, we broke a record for the warmest day on several days and September was the warmest month of the year — warmer than even July and August –for the first time since records were kept. We had a string of days when we saw temperatures 30 degrees above normal highs. While this was a record-setting year, the past few decades have had warmer winters. We couldn’t even skate on the local lake for a long period in the winter because it was so warm and the ice wasn’t thick enough. So, it is most definitely warmer throughout my life. And the farther north we go, the more pronounced this warming gets.

The growing season has shifted so that we get our first hard frost later by a week or so, and the first frost free day earlier. Tree lines are moving, birds that rarely came this far north are now more common, and many stay around for longer and return earlier. Harvest just finished up in the end of November this year, where it usually ends in October.

Now that in itself might be nothing — I could accept that climate shifts on a regular basis from colder to warmer and back over the long term. However, what I find convincing with respect to global warmig is the research on CO2 concentrations in the atmosphere and some of the paleoclimate research on CO2 levels over the longer term. There’s just too much good science behind this to ignore it, given what we are seeing worldwide. So for me, it’s a combination of what I’m witnessing in the real world, the CO2 rising and the links to fossil fuels and the paleoclimate research.

Talking to FOIA people and persuading them-via pathos and ethos to make an exception because they think Steve is being a jerk? Oh heavens nothing wrong with that at all.

I’ve worked on FOI requests in my area and you work closely with the officers to determine what is to be approved and what is to be denied. The officers know the law and guidelines, and I know the field and what information is available. We work together to prepare a response to a FOI request. It is entirely possible that the FOI officers were completely in line with the law.

“I live in the north where warming is having a considerable effect. I’ve been alive for almost five decades and have seen where I live clearly warm up since I was a child.”

Greenaway, I’m sorry for being nitpicky, but this is not any scientifically documented history. It’s an unverified story. It’s poetry. And it’s not global, it’s local. And, of course, if it is warmer, it could be natural. You even state yourself it might be nothing.

In other places, like “Kentucky”, we’ve had an unusually mild summer. I never had to turn my air conditioning on the entire year, when I usually have it on all the time in June, July, and August.

“However, what I find convincing with respect to global warmig is the research on CO2 concentrations in the atmosphere and some of the paleoclimate research on CO2 levels over the longer term. There’s just too much good science behind this to ignore it, given what we are seeing worldwide.”

OK, but could you be specific on what research you find convincing?

Andrew

“Nick Stokes (Comment#25130)-Talking to FOIA people and persuading them-via pathos and ethos to make an exception because they think Steve is being a jerk? Oh heavens nothing wrong with that at all.

Both of you are clueless. There is no “jackass” except to FOIA.”

I think you missed the point. They put a case that the FOI was not in compliance with the law. Those who administer the FOI agreed. You can see a reference later to another abuse of a similar act that is meant to have something to do with credit checking. Steve’s cesspit was evidence enough for the FOI administrators to agree that the requests were not made in the spirit of the law.

@bugs (Comment#25201)

“Steve’s cesspit was evidence enough for the FOI administrators to agree that the requests were not made in the spirit of the law.”

Please can you provide a reference to the point in the regs where this “spirit of the law” is described?

Greenaway, I’m sorry for being nitpicky, but this is not any scientifically documented history. It’s an unverified story. It’s poetry. And it’s not global, it’s local. And, of course, if it is warmer, it could be natural. You even state yourself it might be nothing.

In other places, like “Kentucky”, we’ve had an unusually mild summer. I never had to turn my air conditioning on the entire year, when I usually have it on all the time in June, July, and August.

Thinking that for a phenomenon to be global, it must be the same all over the globe might be a common sense way of looking at it, but it doesn’t fit with the science. Regional response to warming will be different.

I suspect that it is a total waste of my time to respond to you, but I’m trying to be polite so I will assume you really do want to know what I find convincing and are not just waiting to find something to pounce on.

“Thinking that for a phenomenon to be global, it must be the same all over the globe might be a common sense way of looking at it, but it doesn’t fit with the science. Regional response to warming will be different. ”

I didn’t choose the terminology. I am using what proponents use. If Global doesn’t fit with the science, tell me what word to use. Is it Regional Warming now?

Andrew

I didn’t choose the terminology. I am using what proponents use. If Global doesn’t fit with the science, tell me what word to use. Is it Regional Warming now?

You’re being deliberately obtuse or just plain dense.

Next.

“Please can you provide a reference to the point in the regs where this “spirit of the law” is described?”

Clearly there is a provision, otherwise they would not have agreed to deny the FOI application. Rather than blaming the FOI, maybe you should blame what Steve has conjured up. Without that, perhaps they would have had to comply.

Greenaway,

Does Global Warming affect the entire Globe or not?

Simple Yes or No Question. You up to answering it?

Andrew

bugs:

Clearly there is a provision, otherwise they would not have agreed to deny the FOI application

Circular logic much?

I might be, but this isn’t the time or place to do a lecture on global warming 101. I have a science education and have read a lot of the science and while I am not an expert, I am convinced by what I’ve read and I’m not convinced by the attempts by deniers to discredit it. That is all that needs to be said.

Bugs – “Clearly there….” isn’t a reference.

“That is all that needs to be said.” Sorry Greenaway – I was going to offer to second Andre but I hadn’t realised you are a comedy contributor. Are water pistols some time in the afternoon ok with you?

“Circular logic much?”

No. The FOI team and the CRU team are two completely different entities. If the CRU team had denied the release themselve it would have been circular. The FOI team is completely seperate and makes it’s own evaluations. Once again, maybe you should take another look at how climateaudit works. You are just shooting the messenger.

Andrew_KY–
With “KY” in your name, my bets on you when you meet with greenaway in Weehawken.

curious– Are you challenging greenaway to a duel at Weehawken? And here I was expecting it to be Andrew! Water pistols is less dangerous than the historically accurate weapons.

Greenaway,

Does Global Warming affect the entire Globe or not?

Simple Yes or No Question. You up to answering it?”

Yes, it affects each part of the globe differently though. This is to be expected, as existing local weather patterns change. The models at present can’t give accurate representations of what these changes may be due to the size of the grids. The next generation of computers will help with that, they will be able to model clouds more realistically, and have grid sizes that can indicate how local conditions will change.

No duelling from me Lucia – maybe I’m wrong but I thought it was the second’s responsibility to attend to the weapons? Now I see Greenaway is just in it for fun I can hardly allow my man to turn up with a Magnum (or whatever the modern dualist uses..)

“With “KY” in your name, my bets on you when you meet with greenaway in Weehawken.”

“Water pistols is less dangerous than the historically accurate weapons.”

Let’s just say Mr. SuperSoaker and I are…

…good friends? (puts pinky up to lips a la Dr. Evil)

Andrew

Bugs – “The FOI team is completely seperate and makes it’s own evaluations” …. whilst sat on Phil’s knee in front of the monitor.

Any luck on the reference yet?

bugs (Comment#25213) November 28th, 2009 at 8:42 pm

you need to read the mails were jones explains that he had a chat with the FOI team to explain to him the kinds of people who show up at CA. The discussion he had with the FOI team, detailed n the mails is probably actionable. Stay tuned.

“The models at present can’t give accurate representations of what these changes may be”

No kidding. They do look good in bikinis, though.

Oh, wait. You were talking about climate models.

They sound like great models, though. They will be useful before ya know it.

Andrew

Thank you Simon.

OA

bugs:

The FOI team and the CRU team are two completely different entities

Same institute, lots of cross-communication between the groups. Not “completely separate entities” except in Bug’s universe.

It sounds to me from re-reading the emails that the FOI officers checked out CA and understood right away what was going on.

BTW, Section 21 of the UK FOI Act exempts information accessible to the applicant through other means (one thinks of Briffa’s data), Section 22 exempts information that is meant for future publication, and Section 35 excludes information that is used to develop government policy.

I’m thinking there are a lot of exemptions possible.

As someone who has extensive experience in big pharma, it pains me to see the nonsense written by certain people (cough Simon Evans cough) about FDA standards and data disclosure. When a company files a typical NDA, the paper output would fill a small truck (of course filings are electronic today but you should have seen the old days ). The data requirements are beyond rigorous. The FDA is replete with talented epidemiologists whose job is to evaluate study design and reported results in great detail. As a result, it can take the FDA years to review a filing which is why “fast track” review is so coveted. As regards public disclosure of results, most large pharmas now accept the benefits of transparency and publish detailed results on clinicaltrials.gov. See here for the policy statement of the world’s largest pharma: http://www.pfizer.com/research.....ogress.jsp

So, it is rather ignorant for SA to claim that Climate Science is better than other fields. And he may want to check the data availability policy statement of the world’s premier economics journal: “For econometric and simulation papers, the minimum requirement should include the data set(s) and programs used to run the final models, plus a description of how previous intermediate data sets and programs were employed to create the final data set(s).” (http://www.aeaweb.org/aer/data.....olicy.html)

Climate Science should adopt the same rules.

“bugs (Comment#25213) November 28th, 2009 at 8:42 pm

you need to read the mails were jones explains that he had a chat with the FOI team to explain to him the kinds of people who show up at CA. The discussion he had with the FOI team, detailed n the mails is probably actionable. Stay tuned.”

LOL. Private emails that have been stolen and leaked can’t be actionable. Like I said, you are just shooting the messenger. Clean up your own house.

“Climate Science should adopt the same rules.”

http://www.realclimate.org/ind.....-the-data/

Jeez. I don’t know why I bother. Bugs is so full of himself that arguments seem to go in one ear and out the other…

Hey, KY, aren’t you from Ohio?

Some interesting reading for you
http://www.realclimate.org/wp-.....GE-CPN.pdf

Another quote from Mr Storch

“We have advised Steve McIntyre several times that he should write a paper just on this issue, without blending many other aspects into such a paper. He has not done so, it seems. The example of the GRL paper on the EOF problems has demonstrated that he has a chance to publish in peer-reviewed journals.”

http://blogs.nature.com/climat.....stick.html

Now what was that Jones said about McIntyre publishing…..

steven mosher (Comment#25221)
As I said to Andrew above, I am mystified about this. What do you think is actionable? That Jones talked with the FOI people? How can they proceed without talking to him and getting his side of the story? Is it the disparagement in a private email of CA folks? Or Jones speculation about what may have caused the FOI people’s change of sympathy? Or his speculation that there was a change of sympathy? Or his speaking to the VC and Librarian?

Bugs,

Can I take it that you agree the Climate Science should adopt the same data disclosure rules for published papers as is standard in fields such as Economics? Or is the purpose of your link to claim that it already does? (Or that it partially does?)

“Can I take it that you agree the Climate Science should adopt the same data disclosure rules for published papers as is standard in fields such as Economics? Or is the purpose of your link to claim that it already does? (Or that it partially does?)”

You referred to one journal. In the field of economics, information is often jealously garded, and is proprietary. For example, pricing information.

Nick wrote: “That Jones talked with the FOI people? How can they proceed without talking to him and getting his side of the story?”
.
Quite simple really, there is no “side of the story” in FOI requests.
.
“When the FOI requests began here, the FOI person said we had to abide by the requests.”
.
However, “It took a couple of half hour sessions – one at a screen, to convince them otherwise”. Under what circumstances should the FOI officers need “convincing” that they should overturn their decision? Does Jones know the university’s responsibilities under the FOI act better than they do?
.
He continues “showing them what CA was all about.” This clearly suggests that Jones produced motive. However, motive is also irrelevant for the purposes of the FOI act, but not necessarily for human interests in protecting Jones and, by extension, the university, from “hostile” external forces.

Andrew_KY (Comment#25175) November 28th, 2009 at 1:35 pm

Simon,

Yes. Let’s start at the beginning. How was “the world is warmer than it was” demonstrated to you? What happened that caused you to think that, when you didn’t before? Or have you always believed that and cannot recall ever thinking otherwise?

I am persuaded by evidence of the global temperature records, ocean heat content, accelerated sea level rise, accelerated rates of ice cap and glacier mass loss, melting permafrost, shifting species, changing seasonal dates…..

Do you really think the world hasn’t warmed over the post-industrial period?

brid (Comment#25229) November 28th, 2009 at 11:33 pm

I’d already ceded the point with respect to the FDA.

Bugs,

You aren’t serious, are you? You are equating disclosure of practices in for-profit businesses with academic research. This makes no sense. My premise is that the standards of data availability for published papers in Climate Science should be the same as in Economics.

I quoted the data availability policy for the world’s leading economics journal. This standard has now been widely adopted by most of the major journals in the field. Do you believe that climate science papers should be subject to the same standard?

“Hey, KY, aren’t you from Ohio?”

Yes, I am FL. Isn’t that odd? But KY is not far away from me. Neither is IN.

Andrew

bugs (Comment#25236) November 29th, 2009 at 3:20 am

Another quote from Mr Storch

It’s Professor von Storch or Doctor von Storch or Herr ProfessorDoktor von Storch. It’s never Mr. Storch.

“Do you really think the world hasn’t warmed over the post-industrial period?”

That’s just it, Simon. Neither one of us knows that “the world has warmed” over the post industrial period. You THINK it has. I am not going to commit my brain to thinking that it has, until someone shows me that it has.

You trust that a certain group of people know what they are talking about in this area and believe their claims. I don’t. This group making the claims has demonstrated repeatedly that they don’t deserve any trust.

You are free to believe whatever you want, sans conclusive evidence. I hope that you will let me have the same freedom and not try to impose your beliefs on me, which is exactly the opposite of what most warmers are after.

Andrew

Bugs, you are wrong. I quoted the Econometrica replication policy in full some time back. Similar, indeed almost identical, policies are followed by the American Economic Review, Journal of Political Economy. Of course not all journals use the same standard but an idea of the range of journals across several fields that do have some kind of requirements on depositing data can be foudn by following the links from Gary King’s website, here
http://gking.harvard.edu/replication.shtml

Simon:

I am persuaded by evidence of the global temperature records, ocean heat content, accelerated sea level rise, accelerated rates of ice cap and glacier mass loss, melting permafrost, shifting species, changing seasonal dates…..

I agree with all of these as evidence in favor of warming.

I would hasten to say we don’t fully understand all of that warming. If you look at the period 1910-1950, that represented almost as dramatic a warming as the recent warming period (1980-current). In fact when I look at the trends for the annual data for the two periods I find 0.12±0.02°C/decade and 0.16±0.02°C/decade respectively. This gives a difference of 0.4±0.03°C which is barely significant (p < 0.13). Statistically the two warming periods are barely indistinguishable.

Yet we know from the models that the warming from 1910-1940 was almost entirely natural (anthropogenic CO2 and sulfate emissions nearly balanced each other for this period), whereas it is thought that most warming since 1980 is due to anthropogenic CO2 emissions.

In addition, the models don’t explain the early 20th warming period very well. Indeed they are typically off by 0.3°C or so, which is essentially a 3-standard deviation prediction error.

Here’s my summary statement:

“the world is warming, we don’t fully understand why, but man is playing an important role in it.”

The rub is this isn’t strong enough of a statement to force immediate spending of trillions of dollar world-wide. You need something more like “apocalyptic global warming”:

“unless we act now, we face catastrophic and irreversible consequences”

and as far as I’m concerned neither model nor data support this.

Andrew_KY (Comment#25256)

I don’t know what would ’show you’ that the world has warmed, since you seem to think that all sources of evidence are untrustworthy because you judge some individuals to be untrustworthy. That’s a bit of a dialogue-stopper really!

Carrick,

I would hasten to agree that we don’t fully understand….. well, a lot of things, really! We know that we don’t have good enough observations of current forcings, for example, so we are limited in terms of what we can derive from temperature evolution.

“the world is warming, we don’t fully understand why, but man is playing an important role in it.”

I’d agree with that basically, although I might prefer to say “we don’t fully understand the rate”. I agree also that it doesn’t follow from that that we should execute any particular policy response (that doesn’t exclude large-scale mitigation from discussion, but I recognise that it’s not a given, if only because I don’t think it’s going to happen soon anyway!). The problem can be, though, that it is difficult to discuss policy response in this context when, generally, we can’t all agree on a statement such as yours as a reasonable starting point.

DaveJR (Comment#25244)
The FOIA describes all sorts of reasons why it may be that information should not be released. The FOI people can’t possibly decide whether they apply without information from the scientists concerned.

“It’s Professor von Storch or Doctor von Storch or Herr ProfessorDoktor von Storch. It’s never Mr. Storch.”

My apologies to Mr Storch.

Simon Evans (Comment#25247)-”I am persuaded by evidence of the global temperature records, ocean heat content, accelerated sea level rise, accelerated rates of ice cap and glacier mass loss, melting permafrost, shifting species, changing seasonal datesâ�¦..”

With regard to the temperature records, enough has already been said about the problems with them, but it is reasonable to conclude that the are good enough for such a simple qualitative question.

To nitpick, OHC goes back to ca. 1950 so doesn’t really say much about the rest of the century. Again, it would be fair to say that warming before that is very probable.

“accelerated sea level rise”

Tacking on a fifteen year satellite record to a hundred years or so of iffy tide gauges causes the rate of change at the end to, well, change? NO!

The rate, in truth, oscillates. The last 15 years has seen us start to move to more “normal” rate:

http://www.worldclimatereport......e_fig3.JPG

“accelerated rates of ice cap and glacier mass loss”

At one pole and not the other. Glaciers are also really bad thermometers.

“melting permafrost”

Can you guess what this is?

http://www.worldclimatereport......nkhole.JPG

It’s a sinkhole from melted subsurface ice. Circa 1952.

Benninghoff, W.S., 1952. Interaction of vegetation and soil frost phenomena. Arctic, 5(1), 34-43.

“shifting species”

Which species? Because if you are talking about robins in the Arctic (“No word for Robin in Inuit”) for example, V. Stefansson in 1913, in My Life with the Eskimo givens the word Robin as â��Kre-ku-akâ��tu-yokâ�� (Mackenzie Eskimo) and â��Shabâ��wakâ�� (Alaskan Eskimo). But in general, sure, species are moving around. And many are enjoying larger ranges than they had before.

“changing seasonal dates”

White, M.A., K.M. de Beurs, K. Didan, D.W. Inouye, A.D. Richardson, O.P. Jensen, J. O�Keefe, G. Zhang, R.R. Nemani, W.J.D. van Leeuwen, J.F. Brown, A. de Wit, M. Schaepman, X. Lin, M. Dettinger, A. Bailey, J. Kimball, M.D. Schwartz, D.D. Baldocchi, J.T. Lee, W.K. Lauenroth. Intercomparison, interpretation, and assessment of spring phenology in North America estimated from remote sensing for 1982 to 2006. Global Change Biology (in press).

“Trend estimates from the SOS methods as well as measured and modeled plant phenology strongly suggest either no or very geographically limited trends towards earlier spring arrival, although we caution that, for an event such as SOS with high interannual variability, a 25-year SOS record is short for detecting robust trends. Increased greenhouse warming since the late 20th century would seem to argue for increased, not decreased, shifts in spring during our study period, indicating that processes such as succession, changes in community structure, land management, or disturbance may be more important than previously recognized. Seasonal temperature changes may also be linked to a trend reversal in SOS in the early 1990s.”

“Do you really think the world hasnâ��t warmed over the post-industrial period?”

Well, he might not but I do. However, if that was your argument, you have to do better than that.

Andrew,

“accelerated rates of ice cap and glacier mass loss”

At one pole and not the other.

You’re not referencing sea ice, are you? Can we clear this up before discussing further? I am referencing the loss from ice sheets. you’re not suggesting that is “at one pole not the other” are you?

Well, um, yeah, I am suggesting that. Because there has not been an increase in Antarctic snow melt. And because the snow fall is supposed to be increasing, although there isn’t any confirmation of this as yet.

Tedesco M., and A. J. Monaghan, 2009. An updated Antarctic melt record through 2009 and its linkages to high-latitude and tropical climate variability. Geophysical Research Letters, 36, L18502, doi:10.1029/2009GL039186.

Andrew,

Velicogna, I., (2009) Increasing rates of ice mass loss from the Greenland and Antarctic ice sheets revealed by GRACE. Geophysical
Research Letters 36, L19503.

Your thoughts?

Andrew,

Also:

Chen, J. et al., (2006) Antarctic mass rates from GRACE. Geophysical Research Letters 33, L11502.

Cazenave, A. et al., (2009) Sea level budget over 2003-2008: A
reevaluation from GRACE space gravimetry, satellite altimetry
and ARGO. Global and Planetary Change 65, 83-88.

Thoughts?

Nick wrote: “The FOI people can’t possibly decide whether they apply without information from the scientists concerned.”
.
They might need information about the type of data, although the fact it was gathered using public funds and already published is probably all the information they require. They had already determined that the request was in compliance with the act and it took a lot of effort to “convince” them otherwise.
.
Besides, the kind of information they certainly do not require, and which would be against the law if they let it sway their decision, would be information “showing them what CA was all about”. The nature of the individual requesting the information is completely irrelevant for the purposes of the FOI act and yet Jones claims this is exactly what was used to persuade them to change their decision.

Simon:

I’d agree with that basically, although I might prefer to say “we don’t fully understand the rate”. I agree also that it doesn’t follow from that that we should execute any particular policy response (that doesn’t exclude large-scale mitigation from discussion, but I recognise that it’s not a given, if only because I don’t think it’s going to happen soon anyway!). The problem can be, though, that it is difficult to discuss policy response in this context when, generally, we can’t all agree on a statement such as yours as a reasonable starting point.

I like your word choice better. I also agree large-scale mitigation policies need to “stay in play:” If the science eventually nails it down (as I don’t think it has yet) then we need to be ready to act at least. We may not be able to afford 20 years of dithering (word choice apologies to Barrack) at that point.

That said, there is plenty to build consensus on, including alternative energy sources and the like. The major stumbling block is cap-and-trade, which simply put, I don’t think is a viable solution (economically, socially or politically). Going much further down this road is likely to stray into major political land mines, so I’ll behave.

Nick Stokes (Comment#25240) November 29th, 2009 at 4:49 am

If you look at the posts above you will see what Jones said about his meeting with the FOI officers. If you don’t see anything wrong with that then there is nothing I can say to convince you.

The FOI decision is riddled with inconsistencies. They withheld data giving the following reasons.

1. Most of the data is already publically available. (wrong)
2. 2% of the data was covered by confidentiality agreements
that preclude transmission to third parties. (wrong)
3. Releasing this data would harm CRUs ability to get data
from these sources in the future. ( unknown)

BUT:
A. they admit they lost the agreements and yet somehow phil jones remembers the exact conditions for third party disclosure.
You believe mr hide the decline?

B. they forgot about these agreements when they transmitted
the data to peter webster. Opps!
C. They admit to violating these agreements when they
inadvertantly posted the data. Opps!
D. The only agreements they could find are:
1. Agreements with some british territories
2. An agreement with Norway
3. An agreement with Spain which says NOTHING about
third party disclosure
4. An agreement with Bahrain. ( unsigned I recall)

Now, you tell me. Does it sound to you like the FOI office did their job? the FOI law and CRU guidelines specifically state that
the PUBLIC interest CAN trump confidentiality.

Does the FOI office really believe that british territories will
stop sending them data? Did they ask?
Does the FOI office believe phil jones when he says he remembers that there were more confidentiality agreements and he just lost them?
Does the FOI office really believe that Norway will stop sending them data? Did they ask? I won’t even bring up Norways obligations under WMO guidelines.
Did the FOI office even READ the agreement with spain? I know
it’s in spanish, but seriously it says NOTHING about 3rd party disclosure.
Did the FOI office ask Bahrain?

Now I raise these questions about asking the effected third parties for a very important reason. CRU are OBLIGATED BY THEIR GUIDELINES to inform third parties about the status of confidentiality agreements relative to FOI. That is they must inform third parties that any confidentiality agreement may be trumped by FOI. Since CRU accidently posted the data they are also obligated to inform third parties of this disclosure. Did they?

Willis has just scratched the surface on this FOI thing.

That said, I’m not sold on Grace. The modeled effects are much larger than the measured ones.

Even if it’s correct, the long term prediction from climate models is increasing ice mass for Antarctica and Greenland (due to increased inland precipitation), so that would be at odds with the models. The loss also may not be explainable simply in terms of an increased melt rate, so my bet is there is a calibration issue with the satellite and that “bleeding edge of technology” is striking again.

“Now I raise these questions about asking the effected third parties for a very important reason. CRU are OBLIGATED BY THEIR GUIDELINES to inform third parties about the status of confidentiality agreements relative to FOI. That is they must inform third parties that any confidentiality agreement may be trumped by FOI. Since CRU accidently posted the data they are also obligated to inform third parties of this disclosure. Did they?

Willis has just scratched the surface on this FOI thing.”

Are you going to be prosecuting the FOI?

A. they admit they lost the agreements and yet somehow phil jones remembers the exact conditions for third party disclosure.
You believe mr hide the decline?

I imagine everything associated with the FOI request would be indelibly burned into his cortex.

B. they forgot about these agreements when they transmitted
the data to peter webster. Opps!

So, Jones complied when Webster asked for the data? Webster is a PhD in ocean dynamics from MIT with awards and publications to make any academic jealous. Jones refused when a denier like McIntyre with an BA in Math and one research paper in the peer reviewed literature, with a history of only trying to discredit climate science and the scientists doing it?

C. They admit to violating these agreements when they
inadvertantly posted the data. Opps!

Inadvertently means accidentally.

Now, you tell me. Does it sound to you like the FOI office did their job? the FOI law and CRU guidelines specifically state that
the PUBLIC interest CAN trump confidentiality.

I have no reason to doubt that the FOI office did its job and if they didn’t, it’s not Jones’ fault. If all you have is here say, and not the actual documents in which the FOI office explains its judgement, you are just speculating.

As to the public interest, one might respond that perhaps that is exactly that the FOI officers had in mind. You and other CA supporters may have high regard for the blog, but not everyone shares it or believes it is operating in the public interest.

“You and other CA supporters may have high regard for the blog, but not everyone shares it or believes it is operating in the public interest.”

It only took people who had never seen it before half an hour to recognise it for what it is.

#25279 Steve, I’m not asking you to convince me, I’m asking you to inform me. I’m reasonably familiar with the way FOI is administered here, and once you strip out PJ’s occasionally letting his emotions intrude (in a private email), I can’t see what is wrong with the meeting he described.

The key is that FOI is an administrative process. I think you guys confuse it with FBI. Science is not yet a crime. There is an appeal process which can advance to litigation, but at the base level the process is that a University (or whatever) responds to requests advised by the FOI officer on the law, and by its operating officers on the facts. The FOI officers can’t make an independent decision on whether, say, publication would stop the flow of data from country X. They have to work with what the University officers who actually deal with country X tell them. And to find that out, they have to talk to them. If the University, through its officers, has the facts wrong, that will be tested on appeal at some level.

Your response is a whole mishmash of things
1. Propriety of process
2. Correctness of FOI outcome
3. Whether CRU are breaching confidentiality agreements
4. Whether CRU are correctly informing third parties of the effect of FOI

I think it would be better to focus a bit.

“I don’t know what would ’show you’ that the world has warmed, since you seem to think that all sources of evidence are untrustworthy because you judge some individuals to be untrustworthy. That’s a bit of a dialogue-stopper really!”

Simon,

What’s the best evidence that “the world has warmed”? Out of all the evidences that you say exist, just pick the best one and lets look together and see how convincing it is.

Andrew

bugs:

It only took people who had never seen it before half an hour to recognise it for what it is.

More circular logic brought to you from the mind of bugs. Or was that “minds of bugs”?

What’s the best evidence that “the world has warmed”? Out of all the evidences that you say exist, just pick the best one and lets look together and see how convincing it is.

By all means answer if you wish, Simon, but I suspect anything you post would be just written off out of hand.

Andrew:

What’s the best evidence that “the world has warmed”? Out of all the evidences that you say exist, just pick the best one and lets look together and see how convincing it is.

I’m chiming in (cause I can’t help myself) but how about circa 1800, the glaciers were advancing, and now they’re retreating?

Do you maintain the position we’re in the same climactic environment now as in 1800?

More circular logic brought to you from the mind of bugs. Or was that “minds of bugs”?

None of us have read the actual FOI decision — at least I assume none of us have and if you have, please speak up. For some reason, the FOI officer(s) either felt the information was exempted or the request was not justified. They may be wrong, but that will be up to another process to decide.

The rest is just pure speculation on the precise details of why it was turned down.

“More circular logic brought to you from the mind of bugs. Or was that “minds of bugs”?”

They had no prior knowledge of ClimateAudit. They looked at cliamteaudit and made their own conclusions. Nothing circular there.

Carrick,

It’s my understanding that some glaciers are retreating and some are advancing. If you have a comprehensive glacier report you can direct me to, I would certainly read it.

And to answer your question, I have no idea if we are in the same climatic environment we were in in the 1800’s. That’s a complicated question and we would need to define “we” and “climatic environment” and other questions before I could even hope to try and answer that.

Andrew

we would need to define “we”
Now there’s a conundrum for you!

Nick Stokes,

The first “we” is me and Carrick.

The one in quotes needs to be defined.

FYI

Andrew

Andrew:

And to answer your question, I have no idea if we are in the same climatic environment we were in in the 1800’s. That’s a complicated question and we would need to define “we” and “climatic environment” and other questions before I could even hope to try and answer that.

Sounds like you have a bit of background reading to do.

All I have to say.

Greenaway:

The rest is just pure speculation on the precise details of why it was turned down.

Then we agree. Read bugs’ comments with that in mind, and see you agree with his conclusions, given that.

Here’s my summary statement:

The rub is this isn’t strong enough of a statement to force immediate spending of trillions of dollar world-wide. You need something more like “apocalyptic global warming”:

We don’t ever fully understand anything in science.

Simon-my “thoughts” are that:

“The Gravity Recovery And Climate Experiment (GRACE), a joint mission of NASA and the German Space Agency, has been making detailed measurements of Earth’s gravity field since its launch in March 2002.”

Hardly compares to analysis with much longer records.

““The Gravity Recovery And Climate Experiment (GRACE), a joint mission of NASA and the German Space Agency, has been making detailed measurements of Earth’s gravity field since its launch in March 2002.”

Hardly compares to analysis with much longer records.”

It’s too short a time to be conclusive, but it’s one more supporting piece of evidence. Within the error bounds, the ice mass is shrinking, not growing.

http://scienceblogs.com/stoat/.....s_loss.php

bugs:

We don’t ever fully understand anything in science.

M….good point, but think if that in the statistical sense, bugs. No statistically significant anomalies.

In terms of GRACE, again my concern is that the observed ice loss is almost entirely the drift introduced by a modeled correction to the experimental measurement (which itself is much more flat).

Even if there is ice loss, that doesn’t demonstrate GW is act work in the Antarctica. It’s not like the EAIS surface temp ever gets close to 0°C. In fact, the general assertion is that with GW comes increased precipitation and an accumulation of ice on the EAIS (see e.g. the IPCC write up on it).

What are the odds that everything that could be AGW isn’t?

Greenaway (Comment#25302) November 29th, 2009 at 5:24 pm

“None of us have read the actual FOI decision — at least I assume none of us have and if you have, please speak up. For some reason, the FOI officer(s) either felt the information was exempted or the request was not justified. They may be wrong, but that will be up to another process to decide.
The rest is just pure speculation on the precise details of why it was turned down.”

You want the rejection letter for the appeal?

ENVIRONMENTAL INFORMATION REGULATIONS 2004 – INFORMATION REQUEST (Our ref: FOI_09-44; EIR_09-03)

Pursuant to Mr. Palmer’s letter of 21 September 2009 to you regarding the handling of your appeal of 24 July to our response of the same date in regards your FOI request of 26 June 2009, I have undertaken a review of the contents of our file and have spoken with Mr. Palmer and other relevant staff involved in this matter. I apologise for the delay informing you of my decision but we were awaiting the ‘further particulars’ in relation to this matter that you mentioned in your email of 2 September. Having not received such particulars, I have decided to proceed, given the passage of time, with my decision in their absence.

As a result of this investigation, I am satisfied that our overall decision to not disclosethe requested information is correct.

In response to your first point in your email of 24 July regarding the non transmission of data to non-academics, I have concluded that the reference to non-academics was in error and that the status of yourself, or any other requester, is irrelevant to the factors to consider regarding disclosure of the requested information.

Turning to the points you raised in your email of 2 September, you note that other earlier versions of this data are available on the US Department of Energy website and that Dr. Jones had sent an earlier version of the data to you and had mounted it on FTP server.

In regards the information provided to the US Department of Energy, my investigation has revealed that this was done in the early 1990s prior to the imposition of the restrictions now pertaining to the data pursuant to a contractual obligation at the time. Therefore, the analogy you are drawing does not apply to the data that is the subject of this request.

In regards your second point regarding the provision of the data to yourself, and the fact that the information was mounted & left on our FTP site & also provided to Georgia Tech without securing consent of the institutions that provided it, we would, upon reflection, consider this an action that we not choose to take again. However, having made errors in past does not, in our eyes, justify making the same errors again.

I note that in your email of 2 September, you state that your request was ‘for the current version of the data set’ but in your original request, you asked for the subset of data that was sent to Georgia Tech University. I would advise that the many of the same restrictions apply to the full CRUTEM dataset as apply to the subset sent to Georgia Tech, but this analysis and answer is based on your original request.

In regards the substance of the exception claimed under Reg. 12(5)(f), I would maintain the position taken to date. There are restrictions on the release of at least some of the data cited, and our opinion is that any release would be contrary to the agreements, and release would have an adverse effect on those organisations. DEFRA guidance notes that the Aarhus Convention, which contains the origins of the Directive on which the EIRs are based, protects information volunteered by a third party and requires their consent to disclose it. The purpose of the exception is to encourage the free flow of information from private persons or institutions in order to protect the environment where making it available to the public could inhibit that process. To provide information that has a restriction on further transmission on it would not only damage CRU’s ability to secure such information in future, but would also harm the interests of the organisations providing the information, who clearly have an interest in restricting transmission of the information due to the very existence of the restrictions.

Regulation 12(11) requires that we provide as much requested information as is possible outside the coverage of any applicable exception. After consultation with Phil Jones and other relevant staff in regards the nature and composition of the requested dataset, I have concluded that the data is organised in such a way as to make it extremely difficult and time-consuming to segregate the data in the manner that you suggest and would indeed, in our view, amount to an unreasonable diversion of resources from the provision of services for which we, as an institution, are mandated. Further, we would maintain that where no such segregation has, or will occur, we should not release any of the data for fear of breaching such restrictions as do exist.

I would note that we are, however, proceeding with efforts with the international community to secure consent from national meteorological institutions for the release of the information that they provide us with, and it is fully our intention to publish such data where, and when, we have secured such consent. This is in line with guidance from DEFRA that suppliers of volunteered information should be encouraged to consent to release where appropriate, and where it is lacking, such consent can be sought later in response to a particular request or in order to proactively disseminate the information.

In regards our obligation to assess the public interest in applying these exceptions, I am of the opinion that the public interest balance is in favour of non-disclosure of the requested information. As noted above, the public interest in maintaining the flow of information from institutions to CRU, and maintaining good working relations with international organisations, outweighs, in this case, the interest in the release of the data.

We have contacted the Information Commissioners Office in regards this matter and their advice is that if you are still dissatisfied with this response, you should, at this time, exercise your right of appeal to the Information Commissioner.

Yours sincerely
Jonathan Colam-French

Greenaway,

BTW the reject letter above was sent on Nov 18th a day after the link was posted on Airvent ( and also posted on WUWT, where the moderator embargoed the post — see the comments on the post about the CRU-tape letters ) the letter was dated Nov 13th. The last email was dated Nov 12th. This sequence of events lead me to speculate that the file was compiled internally as a result of the FOI process, a process which came to a formal end on the 12th with a letter being written on the 13th, mailed on the 18th.

bugs:

What are the odds that everything that could be AGW isn’t?

At 0%, as I parse your question anyway. The odds that everything that could be AGW (or blamed on it) “is”, is also at 0%. The truth lies between.

bugs:

They had no prior knowledge of ClimateAudit. They looked at cliamteaudit and made their own conclusions. Nothing circular there.

Everything is circular about it. They met and discussed this with the politically powerful professor Phil Jones, and after 30 minutes “made their own conclusions”.

Heh.

Bugs wrote: “They had no prior knowledge of ClimateAudit. They looked at cliamteaudit and made their own conclusions. Nothing circular there.”
.
Is it was your intention to accuse the UEA of lawbreaking or is it just your ignorance in such matters that made you do so?

“Is it was your intention to accuse the UEA of lawbreaking or is it just your ignorance in such matters that made you do so?”

You could always tell McIntyre to smarten himself up.

Bugs wrote: “You could always tell McIntyre to smarten himself up.”
.
So you do believe the UEA broke the law! Remarkable!

DaveJR
So you do believe the UEA broke the law! Remarkable!
http://www.ico.gov.uk/upload/d.....quests.pdf

No, I don’t. Vexatious requests.

“Do you really think the world hasn’t warmed over the post-industrial period?”

That’s just it, Simon. Neither one of us knows that “the world has warmed” over the post industrial period.
.
I agree with that .
If one accepts that the computed variation has been something like 0.8°C in a CENTURY what is 0 for all practical purposes , then only 2 cases are possible :
.
1) The person saying that he has really noticed effects of warming has been so brainwashed that he considers any change as being due to those 0.8°C and invents changes that don’t exist at his place .
2) The person has really noticed something that implies a much greater temperature increase than 0.8°C but then as it is purely local there is approximately an equal number of people who have noticed something that implies a strong temperature DECREASE . Interestingly one rarely meets the latter .
.
Btw this is a very interesting question and I ask it often around me .
My personnal statistics in the place I live is sofar that 89 % of people noticed nothing .
10% think that they have noticed something but after discussion it appears that their “observation” is due to confirmation bias and stands in contradiction with what really happened locally with temperatures that they generally ignore anyway .
However about 30% are irrationnaly convinced that there MUST be people who observed something somewhere .
When I ask where , it is always on a place that is far away and where they have never been
.
You wonder about the 1% that is missing ?
Those are the people with scientific culture , reading climate blogs and giving the right answer
“It is an absurd question . Nobody experiences a global temperature and nobody can observe and remember anything else than local short term variations . The question is one of calculation , not one of observation .”
.
It reminds me a TV report yesterday .
The usual moron was there showing pictures of algea , corals and other microorganisms and pathetically weeping that they will all die (because of the global warming of course) .
Sofar so good .
But then he explains to the hypnotised spectators that they will die because “Their survival is dependent on an extremely fine tuned equilibrium where a tenth of degree is the difference between life and death . Now with man’s ruthless emissions this equilibrium is brutally destroyed . Etc .”
Obviously neither the speaking moron nor the journalists wondered why all those things didn’t die already 100 years ago when the temperatures were different from today .
And from tomorrow for that matter too …

And on a lighter note but on the same subject of “observed” effects .
I have read in a big french newspaper this morning a leading article :
“The taste of wine is dramatically changing with the global warming .”
Followed ONE full page of absurdities and contradictions but :
- all “experts” agree
- it is bad and will be much worse
- it’s apocalyptic even without mentionning polar bears
.
It’s like if there was a fierce competition between the climate “scientists” , warmers , politicians , madmen and journalists at who will say or publish a more outrageous and ridiculous story .
As the french saying goes , “ridiculous doesn’t kill” but it is a pity because else there would be a brutal climate “scientist” extinction these days .
Not that anybody would miss them anyway .

You want the rejection letter for the appeal?

Thanks for that. So your complaint is with the FOI Office and their decision. It seems that the initial request was denied and then that decision was appealed, and that original decision was upheld.

Now, you must move to the next appeal level. Sounds like everything is working just fine, but there is no guarantee that you will get the answer you desire.

If there was no room for denying requests for information, there would be no FOI office unless it was an office of “compliance” in which all such requests were automatically granted and the only work of the office was to ensure that all requests were granted. That seems to me what you are expecting, but it doesn’t work that way. There are legitimate reasons for denying requests.

You and others in your camp don’t agree with the decision. You can always take it to the next level.

The conclusions of the appeal make sense to me:

the public interest balance is in favour of non-disclosure of the requested information. As noted above, the public interest in maintaining the flow of information from institutions to CRU, and maintaining good working relations with international organisations, outweighs, in this case, the interest in the release of the data.

In other words, the FOI office believed it was more important to ensure that relations between the CRU and other nations providing information were maintained by honoring contracts than giving some retired minerals consultant and his cabal the right to dick around with the data and make outlandish and unfounded allegations about hoaxes and frauds.

The usual moron was there showing pictures of algea , corals and other microorganisms and pathetically weeping that they will all die (because of the global warming of course) .

Are these morons “scientists” doing the actual research or were they “activists” who were presenting the case? I want to know if its scientists you hold in such contempt or activists, because if you hold scientists in contempt, that explains a lot.

The person has really noticed something that implies a much greater temperature increase than 0.8°C but then as it is purely local there is approximately an equal number of people who have noticed something that implies a strong temperature DECREASE .

I just read this excerpt showing your reasoning, so you don’t need to bother answering my previous question.

bugs (Comment#25334)-Again, Antarctic Ice Melt is becoming less and less.This is based on a thrity year analysis. 2008 was record low ice melt. GRACE is far to short a dataset to reach any conclusions.

greenaway:

Your capacity to miss the point is boundless. You are indeed the perfect the True Believer and a perfect specimen of the CAGW religion.

Here’s another question for you to miss entirely: When the FOI response letter refers to the “agreement” with other nations, what exactly is that? Where is that document? How come no one was able to produce it when the existence of the “agreement” was itself the subject of a request?

Did Honduras expressly say that temperature measurements taken there can only be shared with friends of Michael Mann?

Was Phil Jones in violation of the agreement when he “lost” all the raw data?

Greenaway, you are apparently the only person on the planet who still buys the bogus “relationship with other nations” ploy by Phil Jones, obediently carried out by his underlings in response to the FOI requests. Again, congratulations.

Pathetic on my part on #25373 above … “miss the point” rather than “point the miss” Not sure the edits took.

George Tobin (Comment#25374) November 30th, 2009 at 8:24 am

The edits “took” (for some reason they do not show up on your computer monitor until somebody else posts on the same thread.

But I honestly thought saying “point the miss” was by design, and quite humorous.

Greenaway,

You wrote:

“In other words, the FOI office believed it was more important to ensure that relations between the CRU and other nations providing information were maintained by honoring contracts than giving some retired minerals consultant and his cabal the right to dick around with the data and make outlandish and unfounded allegations about hoaxes and frauds.”

I’m glad you didnt avail yourself of all the facts. In previous FOI decisions CRU have said the following.

1. On the word of Jones they BELIEVE that these agreements exist. We asked that they search for these agreements and they came up with virtually nothing. Do you see the problem with this? Jones claims that these agreements were made. At first he told them he remember that the agreements said the information could only be transmitted to other academics ( like peter webster). Then we had an academic request the information. And those were denied. Basically jones is making shit up. The other problem is this. by relying on missing agreements to render a decision the FOI office encourages losing documents. You see how? When McIntyre first asked jones said that the agreements precluded release to non academics because he had been caught shiping the data to webster. And the only way he could prevent that was by inventing this condition in a document that he lost.
As the FOI office writes: “In response to your first point in your email of 24 July regarding the non transmission of data to non-academics, I have concluded that the reference to non-academics was in error and that the status of yourself, or any other requester, is irrelevant to the factors to consider regarding disclosure of the requested information.”

THE REFERENCE TO NON ACADEMICS WAS IN ERROR. what that means is this. In a earlier rejection the FOI office talked to jones and jones told them he remember the agreements precludeed release to non academics. When we questioned this on appeal, they decided “the reference” was wrong. The problem remains what were the conditions of these missing documents? Did they expire? were they legally binding? tons of questions BUT jones is there to fill in the details by memory. trust him to do that?

2. By looking at the station list ( list of station names– not the data) and by looking at a leaked file we determined that a VERY SMALL percentage of the data could in fact be covered by agreements. That is over 98% of cru data is freely available at GHCN. In fact in previous requests CRU have argued that ALL the data is freely available at GHCN, but then jones “remembered” that this wasnt the case. Anyways, we know that the number of countries that MIGHT have agreements is very small.

3. CRU did publish the agreements that they could find.

A. some agreements with BRITISH TERRITORIES. And now they argue on appeal that if they give us the data, these british territories may not give them data in the future. Well, those territories are subject to the same WMO guidelines that REQUIRE THEM to cough up the data.

B. An agreement with Norway. This agreement said very little about giving the data to others.

C. an Agreement with Spain. This agreement HAD NOTHING in it about transmitting to third parties.

D. an agreement with Bahrain.

So there you have it. CRU is afraid that these data sources will be comprimised. Does that make any sense to you?

Finally, the FOI office cannot take into consideration ( as it did) whether or not McIntyre would use the data to make outrageous claims. the FOI doesnt permit such a consideration. The public has a right to the information. This right is assumed to have priority.

TomVonk:

If one accepts that the computed variation has been something like 0.8°C in a CENTURY what is 0 for all practical purposes , then only 2 cases are possible :

That’s global average, but it’s not uniform: The effect is larger over land than ocean. The effect is larger as you go farther north. There is seasonal variation in the effect.

And the effects have observable over a lifetime: For example, the shifting North of climate zones in the United States. I have lived long enough to have seen that. Any gardner will have noticed a change since 1970.

Andrew:

Again, Antarctic Ice Melt is becoming less and less

Which actually is consistent with the models for a warming Earth, due to an increase in precipitation. (Though technically what is happening is increased melt close to the coastline more than offset by increased accumulation inland.)

The same effect is expected for the Greenland interior too.

Shifting climate maps.

I’m not complaining.

Carrick,

Thanks for the info. I’ll put the little ice age book on my list.

However, your arborday.org link containes this:

“Global Warming Information Resources
Helpful links to trusted resources.”

Which includes:

“Intergovernmental Panel on Climate Change is currently starting to outline its Fifth Assessment Report which will be finalized in 2014.

The US Environmental Protection Agency offers comprehensive information on the issue of climate change.

NOAA’s National Climatic Data Center (NCDC), the world’s largest active archive of weather data, provides answers to frequently asked questions on global warming.

Research at the NASA Goddard Institute for Space Studies (GISS), at Columbia University in New York City, emphasizes a broad study of global climate change.

This Web site for An Inconvenient Truth provides references to scientific evidence and research on global warming.”

Advertised as trusted resources? Are you freakin’ kidding me?

Andrew

Carrick (Comment#25378)-Yes, yes it is. It is NOT consistent with the belief that Antarctica is losing ice mass at an every faster pace, which was implied by Simon.

Your capacity to miss the point is boundless. You are indeed the perfect the True Believer and a perfect specimen of the CAGW religion.

Perhaps it is you who is unwilling or unable to appreciate that others see the issue differently and come to different conclusions.

By looking at the station list ( list of station names– not the data) and by looking at a leaked file we determined that a VERY SMALL percentage of the data could in fact be covered by agreements. That is over 98% of cru data is freely available at GHCN. …CRU did publish the agreements that they could find.
A. some agreements with BRITISH TERRITORIES. …B. An agreement with Norway. …D. an agreement with Bahrain.

It doesn’t matter if you think the agreements with other countries are unimportant — if they exist, and apparently they do, under the law, they are legitimate grounds to refuse releasing that data. It seems as if the FOI office made its ruling on that basis.

So, please correct me if I’m misunderstanding — 98% of the data is available to everyone, Steve McIntyre included. 2% of the data covered by agreements with other countries is not available.

You refuse to accept the FOI office’s ruling because you don’t think those agreements are valid or important enough? So, you think the law can be ignored because you think those countries aren’t important enough?

Andrew_KY, you have to consider the source, but the shift in planting zones is sound data.

I think I said it straight rather than implying it, Andrew!

I understand your reservation regarding the length of the GRACE records. And there are other reservations as well, of course – PGR measurement, etc. It is good that we have these observations now, and our knowledge will increase going forward.

Carrick,

“the (fill in here) …is sound data”

I’ve heard similar claims before.

Forgive me if I’m still a Lil’ Skeptical.

Andrew

Andrew_KY, all I was drawing your attention to are 1990 and current plant zone maps.

If you feel disinterested to believe those are real…you’ve got issues.

Carrick,

“If you feel disinterested to believe those are real…you’ve got issues.”

This doesn’t sound so forgiving.

Andrew

LOL, just kidding Andrew.

Another source for the same data.

Carrick
.
That’s global average, but it’s not uniform: The effect is larger over land than ocean. The effect is larger as you go farther north. There is seasonal variation in the effect.

And the effects have observable over a lifetime: For example, the shifting North of climate zones in the United States. I have lived long enough to have seen that. Any gardner will have noticed a change since 1970.
.
That was precisely my point that Greenpeace of course totally missed .
You do not observe 0.8° C per century . NOBODY does . 0.8 is 0 for all practical purposes .
What you , perhaps , can observe are at least 1 or 2 °C per few decades what is equivalent to some 4-8°C per century . So about 5-10 times the global average variation .
But , and this was the important point , if you happen to live in a place where the trend was like 4-8°C/century then NECESSARILY there is an equal amount of people who should have noticed a decreasing trend by about the same amount .
It is obviously necessary because the global average may only increase by 0.8 .
So the right formulation , especially in your example should be “Over one or two particular decades at a particular place I have observed things that are consistent with an important temperature variation .”
Again very obviously that doesn’ mean anything for the whole USA let alone the Earth during a century .
.
I have been coming to Boston area in a regular way since the mid 70ies and I can contradict your observation by saying that when I have been there this year , I didn’t see any “climatic” difference to what I remember from 30 years ago .
.
As for Europe (much more north than USA) , I have a garden too and I have observed nothing . It still rains , summers are hot and it snows in winter . The trees are on their place , a bit bigger after 30 years and cherries still taste the same . Last year was very cold and 1976 was one of the hottest years with 2003 I remember . Ah yes , bugs , worms and moles are doing the same mischief . Etc .
.
The point being , I repeat it again for slow people like Greenpeace – if the average varied by about 0 over a century , then there is about an equal amount of warming and cooling local “observations” .
If only large warming “observations” took place then either the observations or the global variation would be fake .
I add that only a decadal “observation” would be irrelevant anyway because short term is supposed to be weather .

Tom,

…if you happen to live in a place where the trend was like 4-8°C/century then NECESSARILY there is an equal amount of people who should have noticed a decreasing trend by about the same amount .

I don’t think so. It is a matter of the area of greater warming compared to the area of lesser warming or cooling (and also a matter of population distribution, of course). For a start, land area is less than ocean area, so your 0.8C/century warming already embraces greater warming over a smaller area.

You suggest “at least 1 or 2 °C per few decades” would perhaps be observable. This GISS page representing maps of their ‘CSCI Index’ gives a view of evolution over four decades, with various regions accumulating gain to your suggested level and with no equivalent areas of equal cooling -

http://data.giss.nasa.gov/csci/global/

Tom, you are of course correct in your assertion that nobody would (likely) notice a 0.8C/century, but that isn’t the effect over land that’s the effect over land+ocean. So that 0.8C/century isn’t a relative metric.

Over land (globally), there’s been roughly a 1°C increase since 1980 (similar value for the continental US). So that’s the number we need to be discussing.

Secondly, according to the models, AGW only kicked in roughly 1980, so any changes prior to that would presumably be due to natural forcings.

Third, the effect of AGW isn’t symmetric over a day and season. If, for example, increased CO2 resulted in increased humidity as hypothesized, then increased nocturnal moisture content equals less heat loss equals warmer night time temperatures on cloudless nights. The planting zones are all about the maximum number of days below a given temperature, and that is generally set by cloudless winter nights, so you could expect a zone shifting more rapidly than would be expected by change in annual mean temperature for a region.

I’m not saying this is the explanation for what is happening (though it’s plausible), I’m simply pointing out that mean numbers can hide a lot of other effects, that are in fact observable by individual human beings.

Tom, you are of course correct in your assertion that nobody would (likely) notice a 0.8C/century,
.
Yes and that is really basically my point (without the “likely”) .
Andrew has a valid a legitimate point by saying that there is no local short term real observation which would “demonstrate” a global and long term effect .
By definition .
There is no “need” to start the observations at an arbitrary point like 1980 and to “observe” just for an arbitrary time scale like 20 years .
And even for that arbitrary time frame I didn’t observe the same thing in my place in Europe as somebody else somewhere else .
Even the warmers are telling us that below 30 years it’s just “weather” …
.
Mind me , I am not telling that the Earth is not warming .
It is trivially so and will continue for thousands of years because the Earth is very cold and just came out of the last glaciation .
It will go on warming with accelerations , stops and temporary coolings untill the next glaciation kicks in .
I am just telling that :
.
a) people can’t infer global effect from their own per definition local observation and that it is a property of an average that for a very weak global variation about half of the observations would support the global variation and a half would contradict it . Actually as the majority of the people live near seas in tropical and temperate regions , the majority should make contradicting “observations”
.
b)
There is no “natural” time scale for the climate . I consider fundamentally wrong the paradigm dominating the actual climate science according to which the system is in equilibrium and everything are just “small” (linear) perturbations .
On the contrary the system is and will always be violently out of equilibrium and its dynamical orbit has nothing to do with some “return” to equilibrium be it radiative or otherwise .
Even if 90 % of actual climate scientists apparently ignore non linear dynamics , there is an increasing number (like Tsonis) who study the system as what it is – an out of equilibrium dissipative and chaotic system .
The corollary is that it stays chaotic at all time scales , if there are trends at a given time scale then those trends can’t be extrapolated to different time scales and the system is dominated by generalized negative feedbacks which keep the system in its attractor .
.
That’s why I can believe you when you tell me about your local observation on a given time scale but I know that it is irrelevant to what the system will do on other time (and space !) scales .