Readers will recall I posted the GISSTemp October Anomaly on Nov. 10, and mentioned that it was always fun to check the Google cache for changes. Little did I know they values would change twice between Nov 10 and Nov 12.
So, now I find myself posting the values I showed Nov. 10 and the newer values today. The only real change from Nov 10 is the October value, which decreased precipitously: it dropped 0.2C.
To provide context for this magnitude of drop, I plotted the Jan 2001-Oct 2008 GISS Land/Ocean anomalies from both the Nov 10 and the Nov 12 updates. These are shown below:
The Nov 10 and 12 data are indistinguishable except for the October anomaly. The 0.78C, which would have been a record for October is circled in green; the 0.58 C is circled in blue. It’s easy to see the 0.2 C is substantial in light of the overall scatter.
As I noted in an early post, Siberia was thought to be warm– just not burning hot as in the first GISS update. Due to warmish conditions overall, October anomaly was high enough to start making the GISSTemp Jan 2001-now trend rise relative to the value in September, though the trend remains slightly negative.
For those curious what happened to the anomalies from June-September, I took snapshots of the recent values. GISS Temps first attempt at an October update is on top; the September values are below them. At the very bottom, I included GISS’s second October update.
- June’s anomaly rose from 0.29 C to 0.32 C.
- July’s anomaly dropped from 0.53C to 0.52 C.
- August’s anomaly dropped from 0.50 C to 0.39 C.
- Septembers anomaly rose from 0.49 C to 0.50 C.
- The current October anomaly is 0.58C down from 0.78C reported on Nov 10.
I suspect the October anomaly is still unstable. The Brownie bet to guess the October anomaly to be reported in December is still on!
Lucia, you are missing the point. This is all preliminary data, it is expected to change. The problem is that GISS does little if anything to identify this as preliminary data.
I agree. On Wattsupwiththat it’s called a “debacle”. Get out of here, this is supposed to be raw data so the term “debacle” is entirely misplaced.
It is this craving for the other sides mishaps that makes me utterly distrust *both* sides in the discussion.
Mike —
Rest assured– I am well aware that GISS does nothing to check– see the previous posts yesterday.
P– The GISS product is not “raw” data. The thermometer readings are rasw data. GISSTemp is a heavily processed data product. That’s neither a slam nor an endorsement; it just is what it is. UAH, RSS, and HadCrut3 are all heavily processed data products.
GISS didn’t need to get this data out on Nov 10. Their system doesn’t have sufficient QA checks. They appear to have let this out without doing remotely reasonable reality checks, and they got egg on their face.
At RealClimate.org
# John Philip Says:
Per Mike C’s comment – “Preliminary” means that only 10% of the stations to date reported false data! – AND only 46% of stations had reported in.
By errors “significantly higher than one would expect” what error rate does Gavin expect?
GHCN fix not complete. See: http://www.climateaudit.org/?p=4332#comment-311565
Sigh. Sad to see in comments @ WUWT and CA that people are surprised at the Canadian and Australian anomalies… I’m not as you well know.
Yes. We expected that. I discussed that in the brownie bet post.
Canada and Australia were warm. The month may well have been warm. It just wasn’t as smokin’ hot in Siberia as GISS reported.
“GISS didn’t need to get this data out on Nov 10. Their system doesn’t have sufficient QA checks. They appear to have let this out without doing remotely reasonable reality checks, and they got egg on their face.”
Presumably they rely on NOAA to do the quality control on the GHCN data which they provide, after all they make the following claim on their website: http://www.ncdc.noaa.gov/oa/climate/ghcn-monthly/index.php
Both historical and near-real-time GHCN data undergo rigorous quality assurance reviews. These reviews include preprocessing checks on source data, time series checks that identify spurious changes in the mean and variance, spatial comparisons that verify the accuracy of the climatological mean and the seasonal cycle, and neighbor checks that identify outliers from both a serial and a spatial perspective.
So it appears that NOAA dropped the ball and GISS is carrying the can for them.
I would hope that the readers of WUWT and CA would take the time to check out the Blackboard as well. Oh, well. I provided a link in the comments there.
Phil. (Comment#6616) November 13th, 2008 at 9:51 am
I continue to see this line of argument in these discussions.
It is wrong. Or, as some like to say, it isn’t even wrong.
When I sign my name to a piece of work that means that I am responsible. I don’t get to go back later and say, Ooops, I forgot; but that’s not my fault.
This argument plus the argument that it’s ok to rely on people not related in any ways whatsoever with the work for Quality checking is extremely disconcerting. Many of these people are involved as a hobby(?), part time, I’m kind of curious level. Oh, and we aren’t Certified Climatologists so we’re not qualified to comment on any aspects of the Global Climate Crisis.
If this is the true situation, kindly explain to me how we can have any confidence, any at all, on any of the GISS/NASA work. Are we to assume that these informal processes (and that nomenclature is far far too generous) have been applied to everything GISS/NASA has done for decades?
You can’t be serious.
Lucia – It has nothing to do with QC, it has to do with correcting errors and waiting for the data that comes in late every month. The data is usually not even finalized for about 6 or 7 months, the problem is that Hansen does little to advise the public that all of this data, every month, and extending for several months, is preliminary. I was able to easily recognize the problem with the data over Canada. It was gray on Monday and generally red today. That area of the world is late every month.
Few in this discussion seem to understand how the data reporting system works. If Hansen et al download the data on Friday, process it over the weekend then publish it on Monday, then data that was reported between Friday and Monday by GHCN will be different.
The notion that Hansen has a QC problem is certainly relevant when it comes to the condition of 90 percent of the surface stations being lousy. But when it comes to running a script to look for repetitive monthly temperatures at individual stations, I cannot agree, I regularly see month to month temperatures at stations that are the same, it is not unusual. This one was easy for auditors to find because the wrong month was reported for many stations in a part of the world that is known for significant temperature swings at that time of the year.
The major flaws in GISTEMP remain the significant guesswork he does related to poor spatial resolution ie; Antarctic temperatures before 1948, Arctic temperatures above 80deg north, Central Africa and South America where no surface stations exist with any significant length of record (this includes UHI in networks that have low density.)
Dan Hughes (Comment#6618) November 13th, 2008 at 10:11 am
Phil. (Comment#6616) November 13th, 2008 at 9:51 am
I continue to see this line of argument in these discussions.
It is wrong. Or, as some like to say, it isn’t even wrong.
When I sign my name to a piece of work that means that I am responsible. I don’t get to go back later and say, Ooops, I forgot; but that’s not my fault.
So you agree that NOAA should come clean regarding the failure of their advertised QC on the data they publish?
Mike C (Comment#6619) November 13th, 2008 at 10:20 am
If the report is not for Month XXX, then it should not be labeled to be the report for Month XXX. It should not be labeled to be for Month XXX until it is for Month XXX.
Phil. (Comment#6620) November 13th, 2008 at 10:41 am
How has something so simple gotten so screwed up. Everyone all along the chain is responsible for correctly and accurately reporting what they have done. And for providing an accurate nomenclature for what the results are.
This stuff is so fundamental that the current situation is almost beyond comprehension.
Think about it Dan, it’s essentially a typoe. It’s been fixed. It’s time to move on and have a discussion about what is really wrong with GISTEMP.
Mike C, It hasn’t been fixed. It seems unbelievable, but the ‘corrected’ data is still wrong.
Lucia, you’ll need a take 3!
See CA
One issue that may come out into the open is how GHCN collects (or doesn’t collect) data. Gavin says that Irkutsk for example didn’t “report” September data. They did. You can find it on the internet. Why GHCN didn’t include September Irkutsk data puzzles me and I’m sure will puzzle others as they become aware of it.
This is part of a more general problem that I’ve discussed from time to time at CA- the filaure of either NASA or NOAA to update the huge number of world stations. I discussed this last year in the context of Wellington NZ, which they’ve lost track of since 1990, but there are hundreds of stations that haven’t been updated by GHCN (or GISS) for nearly 20 years.
GISS can point fingers at GHCN, I guess, but surely they have some responsibility for failing to demand that GHCN update these stations if they are using GHCN as a supplier.
Phil. (Comment#6616)–
Both NOAA and GISS dropped their respective balls. GISS is getting more flak for a number or reasons. One is that Gavin is blogging and excusing GISS’s role. No one at NOAA is publiclly explaining how this is not their fault, how they don’t have enough money, how the mistake was unimportant, how the real problem is (evidently) that skeptics don’t download Model E and provide Gavin with bug reports.
However, you are right that, it wouldn’t hurt for NOAA/GHCN to step forward and tell people they were part of the problem.
But the fact both contributed to the problem doesn’t absolve GISS. They choose to rely on GHCN and don’t do cursory checks.
What would you say to a graduate who didn’t reality check data before analyzing it? It’s standard practice for people to do some routine checks. (Example: Did I put a wire a little rotating gizmo and make sure the LDA part of the off the shelf PDPA system Bill Bahalo sold us measured the correct velocity? Yes. Did I reality check the particle size measurements by using known size particles? Or did I rely on a peer reviewed article by Phil “what was last name again” to assure me the whole theory was ok? No. I trust that Phil whats-his-name and all that… but I also tested with know size particles. Unless you tell me other wise, I will assume Phil would have done the same.)
fred
Don’t worry. I get plenty of page views.
Mike C Of course there are other problems with GISStemp. But this one is a biggie. Gavin’s explanations seem illuminationg. It appears he thinks the significance of this is…. what? A good time to kvetch because some people think GISSTemp has poor quality control?
PaulM– Just back from errands. I guess I’ll go check out CA! I know the numbers will change even if they aren’t totally messed up. GISS anomalies almost never stay the same from month to month. What I’m really wondering is if August will stay at 39/100 , go back to 50/100 or end up somewhere entirely different.
Phil. Giss claim to do ICQ on the NOAA data. Apparent there are some holes in it. No big deal there. Simply say: ” we rely on NOAA for data, we perform our own QC checks, apparently in this case they are not rigorous enough. Since we are smart enough to code GCMS we will be smart enough to patch this IQC hole.”
Instead Gavin steps on his own, err, shoots himself in the foot. twice.
Imaginary GISS statement,
Uhhhhhh, OK, we realize we had a small problem putting the September temperatures in October. Now as everyone knows that data has been corrected and reposted. We also realize that the reposted data also has some, uhhhh, additional little discrepancies, but don’t worry, we will keep adjusting, homogenizing, pasteurizing and reposting the data until Steve McIntyre says it is OK.
Thanks,
GISS QC
Mike–
Pasteurized milk spoils quickly without refrigeration. I think we need the records to be ultra pasteurized. Then it will keep almost indefinitely… as long as we don’t open the package!
Absolutely agree with Lucia and others who point out that BOTH the NOAA and GISS dropped the ball. In this case, NOAA is the data supplier and GISS is the processor/publisher. NOAA is responsible for quality-checking their data, and GISS is responsible for making sure their source is accurate.
In this particular case, the error was so glaring and incongruous with other information available, that all GISS needed was one reasonable person to look at the map of 4-13.7C anomalies across all of Russia and go, “Hmm, that seems odd…let’s make sure that data was accurate.” But apparently that was beyond their capabilities.
Jared–
One of the things I find troubling in Gavin’s public response to this is his inept defense of GISS which consists of whiny rebuttals of argument no one seems to have made. For example here’s a sentence from Gavin’s inline response to Martin Audley
Martin never suggested that automatic traps could catch every single thing. As far as I am aware, no one has suggested any such thing.
What people have suggested is that GISS needs to implement some rudimentary, script driven QA procedures that can at least catch glitches like the one that occurred. That glitch was obvious. Rougly 10% if stations reporting showed no temperature change from September to August (as winter approaches in Siberia). This never happen. Many local anomalies were also unprecendented. The product of probability zero and probability zero is zero!
But instead of permitting reality to dictate his position on this, Gavin keeps wanting to pretend people are insisting GISSs develop automated procedures to find every conceivable error.
For some reason, Gavin also seems to be working hard at shifing nearly all the blame to NOAA (who does share responsibility).
He also seems to be suggesting the conversation is being sustained entirely by people other than him.
Newsflash Gavin: You blog about it, other people will keep talking. Even if fewer people read RC than WUWT, this topic will remain open until at least two weeks after both the following occur:
a) GISSTemp really fixes the problem or
b) Gavin’s post on this topic rolls off the front page of RC.
However, you are right that, it wouldn’t hurt for NOAA/GHCN to step forward and tell people they were part of the problem.
But the fact both contributed to the problem doesn’t absolve GISS. They choose to rely on GHCN and don’t do cursory checks.
Agreed I was responding to the almost universal idea that somehow NASA collects all the data and processing and that they made a mistake. In fact the heavy lifting of data collection, processing, error checking & dissemination is done by NOAA (by all accounts a much more generously funded operation than the Gistemp). Something in NOAA’s process failed and they transmitted a defective product. GISS collected the data and whatever testing they have in place failed to pick up the error so yes they carry some blame, but I can count on the finger of one hand the number of posters I’ve seen who acknowledge NOAA’s fault.
What would you say to a graduate who didn’t reality check data before analyzing it? It’s standard practice for people to do some routine checks. (Example: Did I put a wire a little rotating gizmo and make sure the LDA part of the off the shelf PDPA system Bill Bahalo sold us measured the correct velocity? Yes. Did I reality check the particle size measurements by using known size particles? Or did I rely on a peer reviewed article by Phil “what was last name again†to assure me the whole theory was ok? No. I trust that Phil whats-his-name and all that… but I also tested with know size particles. Unless you tell me other wise, I will assume Phil would have done the same.)
I did indeed, in fact I found a problem in the processing by Will’s PDPA system, which many other users before me had missed, by my being so anal (related to the filter settings as I recall)!
Of course sometimes one does those tests and everything is ok for a long time and then some new situation crops up which you miss because you were lulled into a false sense of security.
As I recall NASA used to take relatively unprocessed data from NOAA and do a lot of the processing themselves, following all the ‘Y2K’ issues I think they shifted to using the more processed product and not duplicating NOAA’s work. Unfortunately that’s come back to bite them.
Phil. (Comment#6636) November 13th, 2008 at 5:45 pm
I and others have said that everyone is responsible for their parts. But I will not absolve GISS/NASA any at all. Their part covers both input and output; as does NOAA’s.
In my industry input files are Independently Verified before a single number is calculated by the codes. Every quantity in the input file is determined to be correct. The auditor is completely independent of the person who developed the input file and is free to question every thing in the file.
And while that does entail costs, look at the cost savings over the past couple of days if GISS/NASA and NOAA had followed the practice. And that’s true in general. Rework costs more than doing it right the first time. All those meetings in which the screwup is the subject are avoided, for one thing.
Phil.,
You say:
Agreed I was responding to the almost universal idea that somehow NASA collects all the data and processing and that they made a mistake. In fact the heavy lifting of data collection, processing, error checking & dissemination is done by NOAA (by all accounts a much more generously funded operation than the Gistemp). Something in NOAA’s process failed and they transmitted a defective product. GISS collected the data and whatever testing they have in place failed to pick up the error so yes they carry some blame, but I can count on the finger of one hand the number of posters I’ve seen who acknowledge NOAA’s fault.
Using one finger of one hand, I count 6 posters that explicitly refer to GHCN and those 6 posters also explicitly refer to NOAA. As I write this there are only eleven folks posting on this thread!
Earl–
In fairness, I think everyone agreeing that NOAA is also to blame said so after Phil brought up the issue.
I think the reason fewer mention NOAA’s contribution vs. GISS’s contribution is that no one from NOAA is posting excuses for the mistake at their blog. Gavin is posting all sorts of rationalizations. Each bizarre rationalization Gavin posts results in people explaining why that excuse doesn’t clear GISS of fault in this matter.
Needless to say, its now in GISS’s interest to get a stable number posted!
Lucia,
I agree that Phil. was the first here to point the finger squarely at NOAA. But I am at a loss as to how he can maintain that there is a “almost universal idea that somehow NASA collects all the data and processing and that they made a mistake”. Unless of course he is referring to many posters at various blogs.
Hi Earl,
I sort of assumed Phil meant many posters at various blogs. Phil and I converse here, and at CA. Phil posts at RC; I read. I’m sure Phil reads WUWT. There are lots of other blogs. It’s true there are more people discussing GISS’s contribution to this mess.
FWIW, if you read the comments, it should be clear that when I refer to “Phil what’s his name” who did theoretical work related to the instrument I used to collect data for my Ph.D. that “Phil what’s his name” is same Phil commenting here. (We’ve never actually met though.)
Yes lucia, that impression was based over reading several sources, as usual here is rather more restrained than most (the civilizing influence of the female presence?)
Since no one else will do it, I would like to take full responsibility for the September/October foulup. I would also like to say that every mistake in the temperature record that our government has presented as a reliable bellweather of catastrophic times ahead, is also my fault. It is my fault that I have stood by time after time, and did nothing, when I knew that a giant hoax was being carried out against the men, women and children of this country and all the countries on this Earth. It is my fault that I have stood silent when friends and co-workers repeated the lies that they had heard so many times. I am sorry. I will do everything in my power from now on to make sure that this deception does not stand. If you can see it in your heart to forgive me and just give me five years, I promise that I will do everything possible to right what has been turned upside down. I apologize.
Mike Bryant wrote: I promise that I will do everything possible to right what has been turned upside down. I apologize.
If only…
Lucia, I don’t know how many people are involved at the heart of the AGW debate on the ‘panic now before we reach a tipping point’ side — you know, those who one could sue if it all turns out to be a jack o’ lantern — but if one could come up with a list there would be a fun brownie bet as to which one eventually starts to hedge his or her position. As the whole affair is just politics, then one would expect the cannier ones to start preparing lines of retreat.
I’ve still not seen this affair in the newspapers, but it’s been the most entertaining couple of days I’ve ever seen on the blogs. Maybe Christopher Booker will have a bit of fun with it in The Sunday Telegraph.
JF
Phil.
Even I behave here. I wouldn’t attribute it to Lucia’s gender. If you ever watched Lucia and Dr. Curry go Hammer and Tong on Dr. Browning over at CA, you’d understand that my good behavior is solidly rooted in fear, not chivalry. She’s got a mean left hook that one.
Can I get more brownies please.
If you ever watched Lucia and Dr. Curry go Hammer and Tong on Dr. Browning over at CA, you’d understand that my good behavior is solidly rooted in fear, not chivalry.
Well it was not so gory (pun intended) and I know that Gerry doesn’t react on form with which Curry tried to compensate her ignorance of Gerry’s papers .
Gerry has an unbeatable solid rock position on the hydrostatic assumption and a sane distaste for people who talk about papers that they have not read and who do not answer precisely formulated questions .
I participated in that thread too but I didn’t see anything gender related or fear inspiring .
In the end Gerry won by abandon when Curry left 🙂
Globally , to get more serious , I think that people who behave do so everywhere and those who don’t , don’t .
To all: Good behavior is enforced by pure evil and some php. The Slow Down Boris plugin was tweaked and applied to two people. (Not Boris. Boris is fine. He just gave me the idea one night when he happened to break his response into a whole string and I woke up to see my mailbox full. I reminded myself climate change wasn’t knitting, and I’d better anticipate the need for troll control. So, if you think the tone is nice here, thank Boris!)
TomVonk,
I arranged that fight
http://www.climateaudit.org/?p=2114#comment-202352
called it before the start , put my quatloos on the math guy,
http://www.climateaudit.org/?p=2114#comment-202404
and was smart enough to stay out of the match once it began
(psst that was a funny thread ). ha it also reminded me that I won a hurricane bet that year. no brownies though.
“GISS is getting more flak for a number or reasons.”
Hansen Derangement Syndrome.
Boris, I’ve noted several times that there is motive hunting and speculation and finger pointing and selective perception and confirmation bias on all sides. Except me and Lucia, we are perefectly rational, albeit horrible spellers. The bottom line again.
1. NOAA appears to have made an error when collating monthlies from dailies
as pointed out by John Goetz. This error should be acknowledged by NOAA and the source code released for public inspection. I’ve asked nicely via email. SteveMC has asked nicely. And I asked Anthony to ask Karl et al in person. Knowing Anthony it was done nicely. many eyes looking at the same code is a GOOD thing. It may be embarrassing to have professional software guys look at your code, but it’s worth it.
2. GISS has made claims about their IQC. Apparently, it wasn’t up to snuff.
That is entirely common. ICQ is a net with many holes and as those holes appear you plug them. One thing is for certain, you have to be very circumspect when talking about “the record”.
3. Given the changes in “the record”, given that these small errors are bound to happen and be discovered over time, given that changes in methodology ( See WUWT today comparing GISTempt circa 1999 versus GISTemp circa 2008) can cause rather substantial changes in “the record” It would be wise to resist the urge to publish press releases that depiect a current year or month as the “hottest” on record or the “coolest” for that matter. make science, not PR.
PS. welcome back Boris