Climate nerd that I am, I have been reading many of the opinion pieces popping up at blogs, forums and online newspapers. I found this bit of crack investigative reporting by a George Marshall at The Guardian UK. A google search indicates the “finding” has been quoted by a few bloggers.
But you could find anything in here. I looked and found lots of references to lunch and fun, 94 to hate, 31 to love. Generally, though, the emails are extremely focused, technical, and, dare I say it, really dull.
It’s true many of the emails are dull. It’s also hardly surprising to find the word “lunch” often in the purloined emails from CRU. After all, it’s no secret scientists eat lunch; many of the emails discuss meeting at lunch to discuss the same sorts of topics we are reading about in the emails that do not discuss lunch.
But what of the words “love” and “hate”? One might think the scientists were a very emotional groups– loving and hating all over the place. Are these scientist spending time sending their sweet hon-hons “billet doux” from wor? Maybe CRU is a regular Peyton Place?
Curiosity getting the better of me, I visited anelegantchaos.org and searched “hate”, “love” and “lunch”. Having done so, I now suspect that George Marshall is not what some might call “detail oriented”– or at least not when it comes to checking his facts.
It is true that if you search the purloined emails for “love” and “hate”, the search tool will return 94 emails containing “hate” and and 31 containing “love”. However, the first file, 848679780.txt does not contain “hate” ;it contains “whatever”. The same holds true for the second file 57600338.txt and the third file. Not wanting to search through all 94 files, I can only report that I suspect nearly all of those contain the word whatever.
The first file that matching “love” contained the word Slovenia as did the second. The third did contain, “( Tim, Phil>and I ) would love to collaborate with you on exploring this issue (and the>role of instrumental predictors)”. (How much you want to bet the eventually discussed the possible collaboration over lunch?)
The first three files matching “lunch” did, indeed, include the word “lunch”. However, one reference to “lunch” may surprise people. Email 959187643 Trevor Davies was asked if he was getting a free lunch out of an oil company, specifically ESSO. In fact, it appeared some climate scientists were willing to dirty their hands with funding from big oil. Oh the horror!!
Those who want to have fun will discover that some distribution lists contain the last name Sexton. Turns out George Marshall is right. You can find all sorts of things in those emails. Oh. Those climate scientists!
I think all previous postings/discussions re GISS temp Hadcrut trends are no longer trustworthy. Also just wait for CA analysis of the data files!
I would hazard a guess that “George” and “Marshall” would also show up!
Having read the piece, I am not surprised at these penetrating insights from Mr. Marshall. It is OK for college sophmores to be clueless, but c’mon its time to grow up.
grep -w (unix command) let’s you search for whole words.
That said there are 13 instances of hate, 16 instances of love, 11 of lunch, 33 of fun and 170905 lines total.
That represents a total of 0.04% of all lines or 0.006% of the 1136704 total words, or 0.07% of the 95306 unique words in all of the emails.
I did this all in about three minutes using standard unix utilities.
wc *.txt
gives character, word and line totals
This nice little command gives the number of unique words:
awk ‘{ for (n= 1; n <= NF; n++) print $n; }' * | tr A-Z a-z | sort -u | wc -l
What a load of cobblers. The old “out of context” defence, relies on readers not actually going to the source documents, the straw man, “there is no worldwide conspiracy”, no, just a conspiracy between the Climate Team etc.
How’s Popsie?
Hahaha. What a terrible article in the Guardian! If that’s all it takes to be a journalist, I should become, like, 4 journalists in my spare time.
K roberts–Popsie’s memory is not too great. BUT, he can remember that he comes here every Sunday for Football! My sister and I marvel!
I confess to having searched for ‘fuck’. (All of the instances were in e-mails by Ed Cook.)
I make it 62 “whatever” and 16 “sulphate” which with 13 “hate” still leaves a residue of 3 unknown “hate” words.
Perhaps we could have a competition for finding them? Anybody have some quatloos for a prize?
Fast moving story. Mistakes will be made. I’m still trying to figure out how to search “whole word only” on that anelegantchaos thing.
Lucia
do you have some form of comprehension problem? You generally go for an insignificant detail, and ignore the actual message. What the opinion piece says is quite correct.
Hank– It may not be possible to search whole word only. The script is probably in php and just does ‘stripos($haystack, $needle)’ If the text “needle” is anywhere in the haystack, it finds it. Otherwise, no. The site doesn’t have a pull down menu for “whole word” and also doesn’t seem to let you add quotes, so you can’t add ” hate “. That script was slapped up and posted fast. (I was planning to do it but worried about the copyright implications. Then I saw it was up!)
bugs
The opinion piece is drivel. The author tries to support his unsupportable claims using a pastiche of meaningless factoids that he things will pass as something approaching a reasoned argument, which it might if read by idiots.
BTW bugs– If you would like me to fisk the article later on, I will gladly do so. However it is so bad that I think it better to follow this advice:
posted at planet 3.0
http://groups.google.com/group/planet30/browse_thread/thread/540d3319ec594a06/4aa590660fd839b3?lnk=gst&q=make+fun+of#4aa590660fd839b3
As it happens, many of the talking points ‘defending’ the contents of the emails are riddled with strawman, red herrings, appeals to emotion, hypothetical possibilities totally unconnected to actual facts and, like this one, mysterious revelations emerging from what seems to be a team of investigators who are on crack.
bugs:
Ah! The old “fake but true” defense. LOL.
Yeah, the other Guardian articles aren’t much to read either. They all seem to start with, “after a viciously coordinated cyber-attack on one of the top most prestigious, eminent, transparent and shiny Scientific Research Institutions in the universe…”
I have found the best way to search the e-mails is:
.
As per a suggestion from Lubos, convert all the *.txt files to one pdf file.
.
1) In DOS, use the CD command to change to the directory where the files are.
.
2) still in DOS, use (after first confirming you are in the right directory, with the DIR command.) You may need to specify the directory to where you want to create the combined file in.
.
Type *.* > hademail.txt
.
3) Convert the hademail.txt file to PDF.
.
4) After converting to PDF, and opening in same, use the find function. You can look for whole words, case sensitive etc.
.
All counts are manual, not machine counts.
.
Using the above, I found 4 instances of “hate” (not counting replication in enclosing the replied-to e-mail), 10 of “love”, 8 of “lunch”, and 3 of “fun”.
.
I haven’t redone the count to confirm, so corrections may need to be made.
.
One interesting one with “fun”, was a reference to ‘ “inventing” December monthly values.’
.
One e-mail may also confirm Wegman’s criticism of the paleo-climate community being too close.
.
One male, writing to another male, closed with the salutation:
.
“Love and kisses”
I also found 4 instances of the shortened version of the word “firetruck”.
Twice by Cook, Once by Mann, and once by Briffa.
The e-mail from Cook is quite illuminating, as regards Cook’s opinion of Bradley, and of the “investment” that Mann and Jones have in previous work.
Hmmm… that sounds complicated. If I weren’t somewhat concerned about hassels related to copyright, I might add pull down menus to a script that searches words in a file and gin something up. (I don’t think there are real copyright issues. I think google won the suit that lets them copy everything and then permits people to search. The net effect was transformative or something like that. But I’m not a legal eagle and I don’t want the possible hassle.)
No, its not complicated. A few seconds to run the DOS commands, and a few for the PDF. The “TYPE *.* > hademail.txt” command puts all the *.txt files, into one larger file, then can be converted to PDF.
.
ummm…not showing my age or anything, but I like DOS….
.
Do you want me to e-mail the full PDF file to you?
No. If I set it up on the web, php would search the text files I already have. The system would be like “anelegantchaos” but with pull down menus so you could do exact matches and more than one term.
I was going to do it… but.. as I said a) copyright worries and b) anelegant put something functional up. So, I figured, “why bother”?
Mostly people who are really curious can search on their own home machines, right?
You could publish, on the web, a PDF of the combined text files, which would then be searchable using the browser embedded PDF tools.
bugs is being intentionally obtuse again. How quaint.
Re: Carrick (Comment#24398)
Carrick, your searching kung-fu appears to be much better than mine, so I have a question for you.
I think these emails were scraped from the CRU email archive using a multiple keyword search string, because there are lots of junk work emails (meeting announcements, travel arrangements, newsletters etc), but no junk personal emails (“honey, don’t forget the milk”).
I have figured out that a search on “briffa climate research data” will retrieve 938 of the total 1073 emails, but cannot figure out the remaining keyword(s) that were used.
Does your kung-fu have a quick way to figure that out?
“But climate scientists have always misunderstood the dynamic of public belief and trust. They assume that belief will be built on their data and that public trust is merited by their authority. With the exception of a few outstanding communicators, they often make no attempt to speak to deeper values or make an emotional connection with the public – indeed they see that as contrary to their professional independence.
Climate change deniers have always understood this. They use language that is designed to appeal to deeper values (such as freedom, independence, progress). The narrative they tell of being determined (and even persecuted) free-thinkers, standing against the tide of oppressive and self-interested conformity is designed to create an aura of integrity and trustworthiness.”
Is correct.
Bugs– It’s true that climate scientists do not understand the dynamic of public believe and trust. After that, it seems false.
sorry Lucia. Reading back, I see that you wanted something that could used by all, on your website.
What I gave, is really only suitable for searches on our own machines, as you suggested.
“You could find anything here”
That’s the silliest thing I’ve ever heard. It isn’t even a proper fallacy, it’s an Ignoratio elenchi. That one could find “anything” does not oblige us to ignore what was found-as a particularly outrageous example: Ben Santer contemplating assault and battery of Pat Michaels. How is it relevant that he may or may not have invited someone out to lunch?
Too bad they choose to lie, right Bugs?
I never was much for worshiping authority myself, in any case.
“Too bad they choose to lie, right Bugs?”
They treated McIntyre with the contempt? McIntyre can’t start a topic on one of his ‘audit’s without using a snide crack. He is the epitome of ‘non-science’
bugs (Comment#24442)
November 23rd, 2009 at 8:46 pm
Which I suppose explains why they spent so much time reading his blog and trying to figure out how to not respond.
“Then the Climate Research Unit director, Prof Phil Jones, focuses on one of quotes: “I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years and from 1961 for Keith’s to hide the decline.” For the smear campaign it is only those key words “trick” and “hide” that count – the rest can be made into anything it wants. Jones ignores this and responds with a detailed technical explanation of the passage with reference to the original graphs. It’s like responding to someone calling you a bastard by showing them your birth certificate.”
Jones can say this till he is blue in the face, deniers don’t care. McIntyre himself refers to ‘tricks’, knowing quite well what they are. http://www.climateaudit.org/?p=4274 He is using it in a derogatory sense, he hatest to think that raw data is usefull. But he knows himself what it means, that raw data has to be adjusted to be useful, depending on it’s history. UAH have to use data adjustments on the hallowed UAH Satellite temperature record.
“With the exception of a few outstanding communicators, they often make no attempt to speak to deeper values or make an emotional connection with the public …”
Yuk!
Flesh Crawls!
But fellow travellers do. And quite convincingly it seems. There is an interview somewhere with a climate protestor that had had her deeper self touched with the message that her descendents would never live to grow old unless she acted to stop climate change.
Alex
Speculation:
Maybe the Minister of AGW Propaganda decided after years and Lord only knows how much money, that the great unwashed simply weren’t convinced enough by the Global Warming sales pitch, and the powers that be decided to initiate a “graceful” shutdown.
They made a business decision. The skeptics get what they want (and end to this particular stupidity), and if a few drones are tossed under the bus, who cares? All Blue Light Specials must come to an end sometime. They can always start an After Thanksgiving Sale. It’s not like they are closing up shop anytime soon.
*Thinking out loud*
Andrew
Jones can continue to give unsatisfying explanations of why the “trick” is legit, and people who know it’s not will continue to think it is not legit. Repeating something that is wrong cannot make it right. An author at the guardian complaining that people don’t accept it after it is repeated does not make it right.
“Jones can continue to give unsatisfying explanations of why the “trick†is legit, and people who know it’s not will continue to think it is not legit. Repeating something that is wrong cannot make it right. An author at the guardian complaining that people don’t accept it after it is repeated does not make it right.”
The use of the word ‘trick’ is the same as McIntyre himself used. Raw data has to be made useful, the ‘trick’ is knowing what to do.
bugs, the issue isn’t the word trick, it’s “hide the decline”.
Do we really need to explain why this is a problem?
Get real.
bugs– Stop trying to use explain something you clearly don’t understand.
Monbiot apologies
BBC had files 2 weeks prior
Heads about to roll in Australian politics
More here; http://www.twawki.wordpress.com
“bugs, the issue isn’t the word trick, it’s “hide the declineâ€.
Do we really need to explain why this is a problem? ”
A decline that was caused by non climate issues, attributed to pollution.
“bugs– Stop trying to use explain something you clearly don’t understand.”
I don’t see you have any better understanding than I have.
bugs:
That’s a theory.
You make it sound like it’s the known truth.
Many proxies are sensitive to other climate parameters besides local mean temperature, e.g., rainfall.
You’ve got a million excuses don’t you?
twawki (Comment#24458)
Monbiot’s apology appears to be ironic. That is: He is not apologizing. The link is here. http://www.monbiot.com/archives/2009/11/23/the-knights-carbonic/ The email starting
is clearly made up.
Bugs,
When an honest scientist see’s something in a graph like a downtick he thinks are caused by other factors, he notes the discrepancy and lets the reader decide. He doesn’t use a “trick” to “hide” it and then hopes nobody notices it.
“is clearly made up.”
Ya think?
Joseph:
Sorry I missed this.
Basically you’d have a list of key words for files to include and list of words for files to exclude.
Just think of it as a series of filters that you sequentially apply.
Thanks Lucia – read his satire, but I think if he has to resort to story telling then it shows he has no facts to back up his argument!
kazinski:
Not in Bug’s universe.
It’s perfectly OK to employ a “trick” like this, not tell the people you have done it, indeed make every effort to hide it, then expect them to have been fully informed of what you have done.
Completely honest.
Twaki–
I’m not sure. I’m a bit confused. Is it all sincere until the sharp transition to the letter? If so, I agree with him — but I find the structure ambiguous as it makes me worry that he thought he was being ironic from the start.
But, I suspect he is sincere, and then switches to irony.
Since this is satire, here’s some competition from the master of internet satire:
Iowahawk Geographic: The Secret Life of Climate Researchers
re the ‘hide the decline’ phrase, what do people here object to/reject in Gavin’s explanation for the phrase on realclimate?
“One example is worth mentioning quickly. Phil Jones in discussing the presentation of temperature reconstructions stated that “I’ve just completed Mike’s Nature trick of adding in the real temps to each series for the last 20 years (ie from 1981 onwards) and from 1961 for Keith’s to hide the decline.†The paper in question is the Mann, Bradley and Hughes (1998) Nature paper on the original multiproxy temperature reconstruction, and the ‘trick’ is just to plot the instrumental records along with reconstruction so that the context of the recent warming is clear. Scientists often use the term “trick†to refer to a “a good way to deal with a problemâ€, rather than something that is “secretâ€, and so there is nothing problematic in this at all. As for the ‘decline’, it is well known that Keith Briffa’s maximum latewood tree ring density proxy diverges from the temperature records after 1960 (this is more commonly known as the “divergence problemâ€â€“see e.g. the recent discussion in this paper) and has been discussed in the literature since Briffa et al in Nature in 1998 (Nature, 391, 678-682). Those authors have always recommend not using the post 1960 part of their reconstruction, and so while ‘hiding’ is probably a poor choice of words (since it is ‘hidden’ in plain sight), not using the data in the plot is completely appropriate, as is further research to understand why this happens.”
I read the Monbiot thing as having two parts. The top part he was being genuine in admitting the data needed to be re-analysed. The bottom he was saying that it’s silly to think there was anything more sinister than the age old human vices (e.g. pride, sloth, wrath) at work.
“Les Johnson (Comment#24422) November 23rd, 2009 at 6:01 pm
…
The e-mail from Cook is quite illuminating, as regards Cook’s opinion of Bradley, and of the “investment†that Mann and Jones have in previous work.”
Hey good spot and very understated! Everyone else should check that one 1062592331.txt
Contains the nice paragraph:
“Without trying to prejudice this work, but also because of what I almost think I know to be the case, the results of this study will show that we can probably say a fair bit about below 100 year extra-tropical NH temperature variability (at least as far as we believe the proxy estimates), but honestly know fuck-all about what the over 100 year variability was like with any certainty (i.e. we know with certainty that we know fuck-all).”
His suggestion for “publish, retire, and don’t leave a forwarding address” gives quite an insight what the reaction of the other climate priesthood would be (he’s obviously ignoring the fact it would NEVER GET PAST THEIR PEER-REVIEW!!)
Ok, it was basically a sentence not a paragraph…
David Gould (Comment#24481)
“the ‘hide the decline’ phrase, what do people here object to/reject in Gavin’s explanation for the phrase on realclimate?”
1) Graphs are power visual tools. Removing information from a graph and replacing information with a dry footnote referencing other literature completely changes the impression that a reader will take away. An honest scientist would have left the post 1960 data in and added the footnote explaining why the decline can be ignored.
2) Mann has been quoted as saying no scientist would ever splice temperature measurements onto the end of proxy series yet we now have evidence that he not only does that in leading journals but other scientists copy his ‘trick’.
IOW, Gavin’s explanations do nothing to address the fact that created graphs with the intent to deceive.
See this email conversation involving Ian (Harry) Harris, Tim Osborne, and Phil Jones — another great example of “consensus” science:
http://www.eastangliaemails.com/emails.php?eid=1009&filename=1252090220.txt
If this isn’t cooking the books, then I don’t know what is. Since when does scientific data have to “look good” and scientists need to “be happy with the version we release”? Also, what the hell is a “IDL thingummajig”? Some magic toaster used to make climate change guano?
Just report the facts, that’s all we ask.
None (Comment#24491) November 24th, 2009 at 5:29 am
Thats an interesting email. Ed seems to be doing good science in it knowing darn well that it would tear up the whole field. Does anyone know if they ever completed this work?
Gavin’s reply is typically disingenuous, and the people who really know nothing about these issues, like Bugs, except what Gavin and his cronies tell them, will swallow it without skepticism because that’s what they want to believe.
.
The main issue is not that the decline was hidden, that is a minor point covering up a major problem. It is that the decline happens in the first place. These proxies are used to reconstruct temperatures for 2000 years or so. They are used to “contain” the MWP. To judge a proxy’s “usefulness” they are first calibrated on modern temperatures.
.
What “Hide the decline” is referring to is the fact that some of these proxies cannot be correlated properly to the full modern temperature record, only part of it. The *major* consequence of this is that they cannot be relied upon to reconstruct past temperatures if they cannot properly reconstruct modern temperatures.
.
Despite this problem being known about for many years, the reason for “the decline” still isn’t understood. Until it is, it is impossible to judge what impact it might have on reconstruction of past temperatures.
.
The whole issue, from the small lie (hiding the decline)to the bigger lie it hides (the real uncertainty of historical reconstructions) is designed to deceive, and it works, as Bugs can testify.
The media is being slower than I would like in waking up the the idea that they have been had by a bunch of salesmen-scientists, but I do think they are waking up.
The interesting question for me is this:
Will another insider decide that the message about how rotten things are in AGW promotion land decide to leak sooner rather than later?
David Gould–#8190
When Gavin says “the ‘trick’ is just to plot the instrumental records along with reconstruction so that the context of the recent warming is clear.”., he is either using the term “along with” in a non-standard, misleading way, or he does not understand what Phil did. Jean S. discussed what Phil did, and the code is available. What Phil did was misleading and not acceptable even if other climate scientists have resorted to the same misleading trick.
Hank
I mostly read it this way too… I’m just always a big puzzled when broad irony is suddenly introduced. Irony is hard to do, and it sometimes turns out the writer muffed it, meant the whole thing to be ironic, and people takes parts or serious.
That said: Unless Mombiot comes forward and reveals he never meant the top part…. then we are reading right.
“Ah! The old “fake but true†defense. LOL.”
I was referring to the entirely irrelevant focus on whether or not the word “hate” was used x or y times. May as well debate the number angels on the point of a pin.
bugs–
If you are loving that drivel filled article so well, please tell me this: What point was the author trying to make by even telling us how often “hate” and “love” were used? Either it was always pointless (in which case, it’s worth nothing the case he is making is advanced by hot air) or there was a point but it is unsupported by his “facts”. Which do you prefer?
The article as a whole is drivel. It happens to contain comic relief, and I focused on that. This is a blog– we have now discussed much of the rest ofthe drivel. But, as I said, if you wish the article fisked, that can be done. 🙂
There’s no doubt George Monbiot really means it.
In his latest Guardian article he says
“I believe that the head of the unit, Phil Jones, should now resign. ”
Later on in the comments section he repeats this
“I was too trusting of some of those who provided the evidence I championed. I would have been a better journalist if I had investigated their claims more closely.
“It is exactly for those reasons that Phil Jones should resign. There’s a word for his lack of openness and control of the data: unscientific.”
Lucia-
Irony IS devilish! Your meaning is in opposition to what you’re actually saying. Not good to place the genuine and the ironic tone next to each other. I also strongly object to announcing sarcasm via the “/sarc on” “/sarc off” convention.
Someone once remarked….. It’s funny, occasionally you run into someone who completely misses the irony and gets indignant about some ironic “modest proposal.” On the other hand you never see people make the opposite mistake and say, “gee, maybe Swift’s modest proposal would be a good idea”
Lucia,
Who is Jean S and where did they discuss this issue?
DaveJR,
However, isn’t the divergence problem a well-known issue? Don’t you agree that it would be counterproductive to include a part of a series when that part is known to not accord with the facts? Why would references to articles explaining this not be reasonable?
Jean S is a Finnish statistician and discussed the trick at climate audit here.
Thanks. 🙂
One or two of my posts have been eaten, unfortunately. Darn hackers.
David Gould,
are you a tropical parrot??
at least Bugs can paraphrase his propaganda messages.
kuhnkat
Hmm?
I think the clue about Monbiot’s article is on his web site. The whole thing is headed
“The Knights Carbonic”. I don’t think the second part is an afterthought.
bishop hill has a post, showing that the guardian columnist runs a “charity” that has ~ £700,000 of funding from DEFRA for global warming advocacy