Its been a long road since I wrote a blog post titled UHI in the USA back in 2010, with pitstops at a number of conferences, one internal round of peer review at NOAA and two with the journal, and many helpful comments from folks, but our paper finally is out in the Journal of Geophysical Research (JGR). You can find a non-paywall version here.
All our code and data is available on the NCDC FTP site, and I encourage folks to do their own analysis using it as a starting point if interested. We also have a guest post up at RealClimate that goes into some detail regarding the methods and results.
congrats!
Zeke
The following statement appears somewhere early on (maybe around page 10 of the unnumbered-page MS):
“In any case,the development of a method for identifying and adjusting undocumented shifts appeared to account for more than [blank]of the signal attributed to urban effects on minimum temperatures by Karl et al. [1988].”
Is there a missing number after “for more than”?
Lance,
In the final paper (on the JGR website) its:
“In any case, the development of a method for identifying and adjusting undocumented shifts appeared to account for more than the signal attributed to urban effects on minimum temperatures by Karl et al. [1988].”
Looks like we missed that in the draft version we put up.
yes what an important paper. Now if we could only get a reasonable answer for why the global warmers will not use the running average values for historic temperature and co2, why Lucia refuses to post temperature changes in the arctic when she asserts ice is melting due to global warming and why a 300% positive feedback is reasonable versus a 40% estimate.
Could it be that it explodes their narrative?
0 points are awarded.
Zeke,
Why did you decide to host the paper discussion over at RealClimate as opposed to here at Lucia’s – especially in light of your comment that the paper was the conceived during a blog discussion here at this site?
Will you be monitoring the borehole for comments over at RealClimate or will it be the usual suspects?
nvw,
I expect a discussion over here as well (there is already a lively one at Watt’s place…). For better or worse, RealClimate reaches a larger audience (especially in the media), while the blackboard is better for technical discussions.
Zeke (Comment #110041)
I guess it will start out as that at RC, then usual suspects will take over, happens in any climate blog, but best of luck.
I was banned from RealClimate before I ever posted there (presumably due to my review of Mann’s book). While it may reach a broader audience, I don’t think a site that actively practices censorship should be the primary location of a publication.
Zeke, congratulations on being a lead author of this paper – and with coauthors who are well known in the field. I hope you continue to be accessible to interested participants in these blog discussions.
I have just started reading your paper and can say that I think the author(s) have done a good job in defining the potential problems in doing this analysis. I personally think that UHI should be handled as a part of a more general problem and that is adjusting station series for changing micro climates.
As an aside, I am in the process of putting my suggestions for benchmarking homogenizing algorithms into an email to you.
Congratulations!
nvw/ Brandon —
I don’t have any problems with Zeke posting the main post at RC. He also posts at Yale Climate Forum. Among other things, Zeke worked on this with multiple co-authors and also with Matt Menne which means that it’s good for the work to be posted in some more neutral territory. But even if he’d been sole author, the fact that Zeke often posts here doesn’t mean his work becomes “Blackboard” property.
Lucia,
Thanks! You are also spot-on with the comment about Menne. NOAA/NCDC isn’t big on letting their employees blog, and RC is a much easier sell given that its run by Gavin.
Bravo! Nicely done Zeke.
Congratulations, Zeke, Troy et al. I’m delighted to see this paper out in print. I’m also please to see that, so far, the comments on Anthony’s blog have been constructive in nature.
Good science can make a difference in discourse, so good show and keep up the hard work!
lucia, I don’t have a problem with it being posted at a different blog. My problem is that blog is Real Climate. I don’t think scientific announcements should be made in forums that censor people based upon their views. That said, my view is based on ideals. It may be there is no ideal solution and Real Climate was the best choice.
Regardless, congratulations Zeke!
lucia, I don’t have a problem with it being posted at a different blog. My problem is that blog is Real Climate. I don’t think scientific announcements should be made in forums that censor people based upon their views. That said, my view is based on ideals. It may be there is no ideal solution and Real Climate was the best choice.
Regardless, congratulations Zeke!
I agree that Real Climate is not a very good forum. Its like a re education camp where every “error” is corrected in line by the sarcastic minister of information.
Lucia,
I made no claims about your blog having ownership over Zeke’s work – my comment was addressed to Zeke asking him why he chose RC given its appalling attitude towards censorship. Especially in light of the fact that Zeke credits The Blackboard for the initial discussion that gave rise to the paper, hosting the discussion here would have been a logic place to return the complement.
Zeke,
I notice you have deftly sidestepped my initial question as to who is controlling the thread at RC – is it you or the “usual suspects”.
And absolutely congratulations on your new paper and the exemplary way you made the pay-walled publication, the data and code available.
Thanks Zeke for the great paper and the open discussion!
I guess where I can criticize is your choice of venue. I’m not a big fan of the agenda-driven RealClimate website. Mostly due to the sarcastic Voice of G(od)avin and double-plus-good editorial style – heaping abuse and deleting posts of even respected scientists.
grrrr – wish someone would censor my spelling – that’s compliment above not complement.
Make sure your browser is putting FTP instead of http in front of it.
David Young,
I want to be the Sarcastic Minister of Information (but with capital letters).
Congratulations on your publication! Always an awesome achievement, and an important discussion.
Just to add my congrats, Zeke. It’s a well-written paper that makes sense to me, as fewer and fewer do these days…
Paul
Maximum rural temps are adjusted up from 0.026C per decade (TOBs) to 0.060C per decade in the homogenized NCDC V2.5 (52i) series.
That is alot of adjustment up from TOBs to final. 131%
nvw–
If you click the link you will see that Zeke and Matt’s link back credits a post guest written by Zeke himself. That Zeke thinks The Blackboard is and was a good place to host his own work and ideas in their earliest stages here (see written by Zeke) is complement enough.
Zeke, I have gone through your paper very quickly and I would suggest if you have the time that a summary of the conclusions here might help some of us understand more of the details of this paper.
The reason I say this is that I did an analysis using the USHCN data and the USHCN rural/urban classification a year or so ago to compare the trends for the period 1920-2011 (as I recall) from those two classification with adjusted monthly mean station series. I modeled the data using as independent variables: the latitude, the altitude, and the proximity to large bodies of water of the stations. I found a small but significant difference in trends by these two classifications with the urban trend being larger. I reported those results at this blog. I also noted at that time that most of the USHCN stations are designated as rural and thus the smaller portion of urban stations having on average a larger trend affects the overall trend in the US very little.
I read the conclusions in your paper, Zeke, and was unclear about the statistical significance that you found between the difference in trends from rural and urban stations in the TOB and Adjusted series for the four proxies for rural/urban designation. Just looking at the graphs, I would say that for the period 1895-2010 that for GRUMP, Nightlights and ISA that the urban/rural differences are significant for the adjusted minimum and maximum series.
“The reason I say this is that I did an analysis using the USHCN data and the USHCN rural/urban classification a year or so ago to compare the trends for the period 1920-2011 (as I recall) from those two classification with adjusted monthly mean station series”
the ushcn classification isnt very good. its based on a defunct and not suitable for analysis nightlights file. It basically aliases some rural into the urban class and fails to catch some built up areas.
These missclassifications are a double hit since a station is moved from one class to the other.
Put another way: dont use that classification.
So it’s Zeke’s turn; my variation on Zeke’s question:
What percentage of the warming trend from 1901-2012 in the CONUS is comes from the adjustments versus the raw signal?
Kenneth Fritsch (Comment #110092)
Zeke, since my results look much like yours (from the graphs) I suppose the next step would be to compare the classifications in your four proxies to mine garnered fron USHCN. Our models do, however, from my current reading of your paper, appear to be different.
Zeke, could you redo this graphic taller so the changes are more easily deciphered?
http://wattsupwiththat.files.wordpress.com/2013/02/hausfatheretal_figure.png
A commenter at WUWT made a really good point is that the lines change places around the 1961-1990 pivot point.
Rural goes from .25C warmer than urban in 1910 to urban warmer than rural in 2010 by .10C or so. Thats a .35C change.
How big would the change be if the base period was 1895 to 1925?
Isn’t .35C huge?
Kenneth,
The conclusions section of our paper does a reasonably good job of summarizing the results. As we mention, there is still some significant century-scale warming trend post-homogenization (at least in USHCN v2; less so in USHCN v2.5 which came out after the first draft of the paper was completed). Most of this residual UHI effect occurs pre-1930, when the station network was much more sparse and the ability of pairwise comparisons to detect breakpoints more limited. Here the the relevant part of our conclusion:
There is consistent evidence that urban stations have a systematic bias relative to rural stations throughout the U.S. Historical Climatological Network (USHCN) period of record. This bias has led to an apparent urban warming signal in the unhomogenized data that accounts for approximately 14–21% of the total rise in USHCN minimum temperatures averaged over the conterminous United States (CONUS) for the period since 1895 and 6–9% of the rise over the past 50 years. Homogenization of the monthly temperature data via NCDC’s Pairwise Homogenization Algorithm (PHA) removes the majority of this apparent urban bias, especially over the last 50–80 years. Moreover, results from the PHA using the full set of Coop station series as reference series and using only those series from stations currently classified as rural are broadly consistent, which provides strong evidence that the reduction of the urban warming signal by homogenization is a consequence of the real elimination of an urban warming bias present in the raw data rather than a consequence of simply forcing agreement between urban and rural station trends through a spreading of the urban signal to series from nearby stations.
Because homogenization is largely successful in removing urban bias in the USHCN temperature data, it appears that only about 5% of the period-of-record USHCN version 2 minimum temperature trends across the CONUS (and between 0% and 2% since 1960) can be attributed to local urban influences and, furthermore, that most of this contribution is coming from data for years prior to 1930.
.
Carrick,
Roughly half the CONUS Tavg warming trend is due to adjustments. About half of that is due to TOBs and the remainder is due to homogenization. However, homogenization only warms max temps (and actually slightly cools min temps) relative to the TOBs-only series. As discussed in the earlier Defense of NCDC post, there are a number of lines of evidence (tests on synthetic data, independent replication of results by Berkeley using a different approach and many more stations, etc.) strongly suggesting that these adjustments are justified and do not add a significant amount of bias. Its also worth noting that you get pretty much the same result if you do a manual TOBs adjustment based on documented time-of-obs changes, or if you just let the PHA automatically detect TOBs changes as breakpoints.
.
Kenneth,
Our metadata classifications are up on the NCDC FTP site, if you want to compare.
Congratulations, Zeke and colleagues.
ithcn classification isnt very good. its based on a defunct and not suitable for analysis nightlights file. It basically aliases some rural into the urban class and fails to catch some built up areas.”
Hmm, Kenneth, on second thought it might not be that bad for the US.. It’ll be interesting to see the comparison. You might also considering adding an airports variable to your analysis: lat,alt, distance from water and distance from airport..
GG Zeke!
Zeke: At RC,you concluded that “homogenization does a good job of removing urban-correlated biases”. If I understand correctly from Menne 2009, homogenization in your case corrects only discrete breaks in the temperature record. The development of UHI (on what you call the meso-scale, 10^2-10^4 m) almost certainly is a slow gradual process. How can the bias from a gradual process be removed by a series of corrections at breakpoints in individual station records???
Whatever undocumented changes are responsible for causing sharp breakpoints, they almost certainly are not the cause of UHI. Therefore adjusting discrete breaks does not REMOVE UHI.
A more sensible possibility is that a significant UHI effect doesn’t exist. Instead, urban stations contain more breakpoints (usually from warmer older temperature to cooler newer temperature) than rural stations. When you adjust for those breakpoints, the apparent urban-rural trend difference diminishes. (Unfortunately, we do know that UHI does bias at least some stations.)
Zeke, can I assume from the following statement in your paper that the USHCN classification for Urban and Rural that I used in my analysis and from the GHCN Meta data would be represented by the Satellite Nightlights in your paper? The portion of Rural to Urban stations seems about right.
“3.a. Datasets used to classify station types
Satellite Nightlights
Satelliteâ€derived brightness values associated with the COOP Network stations (including the USHCN) were taken from the Global Radiance Calibrated Nighttime Lights dataset produced by the Earth Observation Group using instruments flown on Defense Meteorological Satellite Program (DMSP) satellites. We used the data from the F16 satellite recorded between 2005â€11â€28 and 2006â€12â€24.”
Zeke, I specifically would like to hear your comments on the 1895-2010 period and with the adjusted series and whether the trend in difference series for Urban and Rural classifications are significantly different than zero. And by classification proxy. I note that you adjust CIs for AR1 autocorrelation, but I would doubt that that be much of correction and particularly so since you dealing with difference series. My eyeballing the results of the plots of the difference in the series in your graphs might be thrown off by your reduction of degrees of freedom in calculating CIs due to using the same Urban station in up to 4 Urban/Rural difference series and using a 5 year MA.
Also is the reduction in DF you used in this situation a common statistical practice?
Frank,
The PHA can pick up trend changes if they are limited to a single station and not present at nearby stations. If you read Menne and Williams 2009, they state rather clearly that “the procedure explicitly looks for both abrupt ‘jumps’ as well as local, unrepresentative trends in the temperature series”. The paper is available here: ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/papers/menne-williams2009.pdf
That said, I wouldn’t be surprised if at least some UHI appears in the form of breakpoints in addition to gradual changes. For example, a previously rural area becomes suburban over time, but a big jump occurs when a large paved area or building is constructed near the sensor.
.
Kenneth,
There is a significant 1895-2010 min urban-rural difference remaining post-homogenization for all urbanity proxies except population growth. It accounts for around 5 percent of the century-scale trend in min temperatures and is primarily clustered in the pre-1930 period.
If you use the same satellite period for DMSP data than you should be using the same data that we are. The cutoff we used was 15, adapted from that used by Hansen.
Neither the station pairing nor spatial gridding approach uses 5-year MAs for CI calculation; all CI are calculated off the unsmoothed monthly data. For station pairing, we cluster standard errors by unique urban station, such that we aren’t overweighting clusters of nearby urban and rural stations in the analysis.
Zeke:
Congratulations. Regarding “Most of this residual UHI effect occurs pre-1930, when the station network was much more sparse and the ability of pairwise comparisons to detect breakpoints more limited. ”
Might this also be due to increases in irrigated agriculture following the reclamation boom in the 1930’s?
Thanks
Zeke (Comment #110107)
The PHA can pick up trend changes if they are limited to a single station and not present at nearby stations. If you read Menne and Williams 2009, they state rather clearly that “the procedure explicitly looks for both abrupt ‘jumps’ as well as local, unrepresentative trends in the temperature seriesâ€.
If the trend changes are sufficiently gradual I do not think the breakpoint functions is going to find them at least not in the simulations I have performed. UHI would I assume occur gradually.
“For example, a previously rural area becomes suburban over time, but a big jump occurs when a large paved area or building is constructed near the sensor.”
I would classify that change as a micro climate change and could occur in any population classified area. I grew up in a rural area and we paved out there – even many years ago when I was growing up.
‘Whatever undocumented changes are responsible for causing sharp breakpoints, they almost certainly are not the cause of UHI. Therefore adjusting discrete breaks does not REMOVE UHI.”
in the Northern hemisphere, UHI has a definite seasonal pattern.
it helps to understand a bit about how UHI develops and the conditions required for it to manifest.
1. It does not happen every day or every month.
2. It reaches maximum values on days that are
a) calm
b) no rain ( also no rain in prior days )
c) cloud free.
3. Summer and Winter have larger effects than other months.
So. you would see no difference between urban and rural in the spring ( for example ) and large differences in the summer and winter months. year over year you might see that seasonal delta
grow, but phenomenlogically when UHI occurs ( cloud free, no rain, no wind ) it appears as an abrupt difference.
That actually gives kenneth and interesting test to do.
take a set of stations within 100km of each other.
calculate a seasonal difference for the station pairs.
using that predict which is urban and which is rural.
in short, if its true that UHI is seasonal ( most every study suggests this ) then seasonal difference should be a “predictor” of urbanity. not perfect, but its an interesting inversion of the problem.
or see how seasonal difference is correlated with nightlights value or ISA value.
Zeke:
Thanks, Zeke. This of course is the gnarly wart in the whole process.
We should all be able to agree that large adjustments are required, and many of us would agree that the adjustments that are made are accurate enough for any residual bias introduced by them to not affect the conclusions… at least for the ensemble (there’s the wart).
Here’s are a couple of related follow-up question:
You said “do not add a significant amount of bias”.
Do you have a bounds on that bias for the trend of the ensemble?
How about the bias in individual elements in the ensemble?
And… how many stations do you have to combine (a SWAG will do here) before you can say that “the amount of bias” introduced by the adjustments is not significant?
[For a reference frame, some groups were not always forthcoming with the magnitude of the adjustments nor how they were being made. AFAIK, Menne’s group has always been very open book about their process, and they are to be commended for that.]
Kenneth Fritsch, that definitely sounds like a microsite issue. Paving and building buildings contributes to UHI, but it does so by adding to a (relatively) large field.
There could be sudden UHI changes if significant construction happened, like times I’ve seen fields get changed to neighborhoods in just a few months. Or if construction introduced/removed a significant windbreak. But that would be uncommon.
‘Zeke, can I assume from the following statement in your paper that the USHCN classification for Urban and Rural that I used in my analysis and from the GHCN Meta data would be represented by the Satellite Nightlights in your paper? The portion of Rural to Urban stations seems about right.”
The GCHN nightlights test is slightly different and not very well documented.
1. they used a prior version of Nightlights from Imhoff and marc ( or the PI i forget which)advises against using that data.
2. They aggregated values from 4 grids (1km) because of station
accuracy issues.
so, the file they used has been superceded and the accuracy of USHCN is fine for 1km nightlights data, there isnt any need to aggregate grid cells as they did for GHCN.
I would heartedly advise anyone who might be interested to use a breakpoints function of your choice and determine the limitations of finding gradual climate changes. If you think you have a handle on the changes that might occur, you could simulate those changes in a series and then estimate breakpoints. I have found the corgen function in R library (ecodist) is a good tool for obtaining correlated series for doing obtaining realistic difference series.
Zeke (Comment #110107)
“Neither the station pairing nor spatial gridding approach uses 5-year MAs for CI calculation; all CI are calculated off the unsmoothed monthly data. For station pairing, we cluster standard errors by unique urban station, such that we aren’t overweighting clusters of nearby urban and rural stations in the analysis.”
What I was referring to here were your graphs and the fact that the nice rather straight lines using a 5 year MA might not be as indicative of statistical significance once the actual data was used and the degrees of freedom due to using the same urban station in several pairs of comparisons were accounted for. I doubt that the adjustment for AR1 would make much difference in the CIs.
I went back to my analysess and found that I had done intitially essentially what you have in your paper except I attempted to account for altitude and proximity to water differences between stations. I had to do the same as you in using paired comparisons with different rural but the same urban station. I had noted that this would require a change in the degrees of freedom in calculating CIs, but I did not have a good reference to how this would be handled by proper statistics. Do you have such a reference?
The paper could have shown the Rural Urban differences and the impact of the adjustments the way you did in this chart (which is probably the most informative one I’ve seen).
http://i81.photobucket.com/albums/j237/hausfath/GRUMPPairs1895-2009.png
I agree with Bill Illis about that chart being informative.
It looks to me like TOBS and other adjustments are about equal the to raw trend.
It’s interesting how big of an effect the adjustments have on rural tmin temperature trend.
That’s be worth investigating to understand the significance.
Carrick,
The figures in our supplementary materials examine that to some extent.
.
Bill Illis,
Figures 1 and 2 are along those lines, though they plot the trend in the urban-rural differences rather than the trends of urban and rural stations separately. The table in our supplementary materials has the trends for all stations and rural stations.
.
Kenneth,
The non-independence of station pairs was something we struggled with when figuring out the best way to calculate CIs. I talked to a number of econometrics folks over at Stanford who suggested clustering standard errors by unique station, and unique urban stations resulted in a smaller amount of pair permutations than unique rural station. I suspect there might be a better way to calculate the CIs (at least for the station pairing approach; the spatial gridding approach doesn’t run into this issue) which would result is slightly larger CIs.
Also, the 5-year running means aren’t intended to be used for inferring statistical significance per se; that is what the confidence intervals on Figs. 1 and 2 are intended to do. Those figures could be recreated using any arbitrary period over which you want to examine trends.
Zeke:
Thanks Zeke. I wasn’t being lazy, I was trying to put you on the spot. 😉
(That is: If I say what I think your supplementary material means, there will always be confusion over whether that’s my interpretation or yours. I’m interested in hearing the primary author’s interpretations, if he’s willing to provide them.)
Carrick,
Table SI.1 has rural tmin trends for both TOBs-adjusted and fully adjusted (for the old v2 and new v2.5 versions of the PHA) for each urbanity proxy. Over the century-scale trend homogenization tends to increase rural min temperature trends very slightly (e.g. 0.006 to 0.008 C per decade). Over the last 60 years it pretty much leaves rural min trends unchanged.
I went back to some of my previous posts and analyses of rural versus urban and rural versus urban and suburban station trends. I originally did something closely aproximating what Zeke did in his paper, but unfortunately I had a coding error in my analysis.
I finally did an Anova using models where I compared trends from rural versus urban and suburban station classifications and had as independent variables in the models proximity to water, altitude and CRN rating. My results were on the same order as what Zeke’s paper shows. The significant differences were between rural versus urban/suburban and not urban versus rural/suburban. I need to go back and revisit those analyses and the variables, with time permitting, and use the latest version of adjusted GHCN data.
Congrats Zeke
Your UHI results provide independent support for Ross McKitrick’s finding of socioeconomic influences in the temperature record.
•McKitrick, Ross R. and Nicolas Nierenberg (2010) Socioeconomic Patterns in Climate Data. Journal of Economic and Social Measurement, 35(3,4) pp. 149-175. DOI 10.3233/JEM-2010-0336.
and contrary to Gavin Schmidt’s finding negligible influence.
Gavin A. Schmidt, Spurious correlations between recent warming and indices of local economic activity, International Journal of Climatology, Volume 29, Issue 14, pages 2041–2048, 30 November 2009
Kenneth Fritsch,
I’d strongly suggest using USHCN over GHCN, given how much more dense the network is. GHCN will also be transitioning over to a new product (GHCN-D) in the future, which contains about 5 times more station data.
.
David,
We didn’t examine the indicies of local economic activity Ross used. We looked at nightlights, pop growth, impermeable surfaces, and administrative boundaries.
Reply to Zeke (Comment #110107)
Zeke wrote: “The PHA can pick up trend changes if they are limited to a single station and not present at nearby stations. If you read Menne and Williams 2009, they state rather clearly that “the procedure explicitly looks for both abrupt ‘jumps’ as well as local, unrepresentative trends in the temperature seriesâ€.”
Frank replies: Thanks for your reply. Table 6 in M&W 2009 shows that the false rate for detecting both step and trend changes with much higher than for detecting step changes. The paragraphs above that table discuss the merits and weaknesses of attempting to detect all five types of modeled artifacts. (The more parameters needed to define a modeled artifacts, the more difficult they were to accurately.) There are several paragraphs on p 1714 the discuss the advantages of looking only for M3-type artifacts, which appear to not correct trend changes. This discussion led me to assume that most or all of the changes included your analysis must have been step changes. Do you have some breakdown that shows how much of the overall correction (that you suggest can be attributed to UHI) was due to step function adjustments and how much was due to trend adjustments? It isn’t a clear to me from M&W 2009 precisely how the various methodS described in this paper were refined into the single method (with code corrections?) used in v2.5.
I’m interested the hypothesis that gradually deteriorating observing conditions at stations might be corrected by maintenance every decade or so. Deterioration would result in a slight bias in the trend (that would be difficult to detect) lasting a decade or two, followed by a step change (that would be easy to detect) restoring the original observing conditions. Some possibilities of deterioration include diminishing reflectivity of the shelter, less effective ventilation, encroaching shadows from growing trees, and gradual nearby development. If maintenance or a minor station move actually restored the original observing conditions and if you only detect the step function change associated with restoring original conditions; then adjustment will bias the trend, not improve it. As best I can tell M&W 2009 doesn’t explicitly attempt to identify this type of “triangular” artifact – one that should not be corrected. FWIW, the hypothesis proves an explanation for why algorithms are detecting so many breakpoints in the record. If deterioration results in more warm than cool biases in the trend, maintenance will produce ore step changes in the cool direction. More adjustments will lower past temperature than raise it.
frank – a point brought up by steve mci when best came out.
Zeke (Comment #110145)
“I’d strongly suggest using USHCN over GHCN, given how much more dense the network is.”
Zeke, I have downloaded the USHCN stations and they numbered around 1218 by way of the ushcn-station.txt file, while for the US 425 country code from GHCN, I obtain 1850 stations.
What am I missing here?
BillC: Are you referring to this ClimateAudit post?
http://climateaudit.org/2011/10/31/best-menne-slices/
This post doesn’t explicitly discuss my hypothesis, but some aspects are clearly illustrated in the graphic from Steve in the comments section (which I don’t recall seeing before).
Congrats Zeke!
Now can you tell me what stations speak the truth? Is there a list?
There is no ‘truth’.
‘
“Now can you tell me what stations speak the truth? Is there a list?”
According to Mosher, this has been tried already, and there is no difference.
http://wattsupwiththat.com/2013/02/13/preliminary-comments-on-hausfather-et-al-2013/
Thanks bugs.
I consider it a tricky business identifying the most pristine stations. They could easily suffer an anti-UHI effect due to de-ruralization. Social-economic factors are also in play. In Appalachia there might be a trend towards fewer hooch stills.
The realscientists at realclimate were having a robust (by their definition) discussion of Zeke’s paper, until someone (Mosher) allegedly attempted to derail the alleged intelligent discussion with his own hypersensitivity. Zeke felt it necessary to admonish his friend so that the realscientists would not think that Zeke did not appreciate the honor he had been given to expose his paper on the official climate science blog. The alleged science of climate is a soap opera.
No, Zeke was doing it of his own accord, not to impress any scientists. If you want to look at who made it a ‘soap opera’, you need look no further than Mosher.
All they did was note that, like the BEST project, we haven’t really learned much that was new. Zeke was addressing concerns raised by skeptics, so his paper is not addressed to the RC scientists as such, but a different audience. Despite all his efforts, I don’t know how many people’s minds he is changing, but he works diligently at it, and does convince some, I think.
Wow. I just found myself agreeing with bugs. That’s about as twisted as a soap opera tale.
Nice job. I still wonder if Anthony’s surface station classifications will produce a larger differential signal, but the conclusions are reasonable enough.
Why publish good work at RC? It has become a mouthpiece for bad science. Hell, every single I read there needs to be re-parsed with toothpicks before I can trust it.
Bugsy says that Zeke was about to make his groveling comment, before he read Mosher’s comment, in which Mosher correctly made the observation that eric had pooh-poohed, if not pooped on Zeke’s paper. Coincidence. Brandon felt compelled to announce that he is on busgy’s side, cause he be mad at Mosher for slapping him around, repeatedly. And I thought Brandon was making progress. Jeff’s comment does not follow the plot line. Makes too much sense. Join us again for the next lame episode of the long-running , well-funded telenovela called the climate science.
Wow Don Monfort. Felt compelled? Mad? Slapped around? I suggest you stop trying you hand at ESP. People might mistake your failure at mind-reading as a failure at reading in general.
I am much better at reading your mind than you are, Brandon.
Felt compelled? yes
Mad? obviously
Slapped around? everybody saw it
You forgot one:
Repeatedly? we have lost count
Fascinating. Tell me Don Monfort, if you’re better at reading my mind than I am, what sort of activity was I participating in when I posted that? Since you say I was mad and felt compelled to post what I did, surely you have some insight.
That isn’t a rhetorical question, but to be safe, I’ll answer it myself. I was sitting in a bar, laughing and having fun cheering a family member on in a dart tournament to win a trip to Vegas for a huge tournament in April (he won the trip!). That doesn’t fit your description very well. Heck, I wouldn’t have been angry at all the whole day except some idiot at the bar started spewing off bigotry. That made me mad. Mosher didn’t.
But hey, maybe I’m full of it, you can read my mind, and Mosher did slap me around. I don’t think you’ll convince many people that’s the case though.
You need to find yourself a girlfriend, Brandon. And a sense of humor. You will probably need to acquire the latter, first.
I think that’s about as low as the discussion is going to go. You’ll have to practice your obsessive mind-reading skills without me.
David hagen
Ross’s work is flawed. The method he used to calculate popuation and population growth is crazy. By his account the population of antartica is 56 Million, seriously. Here is how he calculated populations over time: Take the population of the country.
Divide that population equally into all grid cells. Then he didnt even check to see what happens with territories, so places like st helena get the population of england. And the antartic stations are identified as british so they get the population of england.
Same for all other economic factors.
Zeke, let me preface what I have posted here by stating that for this exercise I claim no statistical rigor or even guarantee that I have not made a coding error (although I did do some doubling checking). It is more of back of the envelop calculations just to let me understand the limitations of using a regression model of the trends for rural, suburban and urban (RSU) GHCN stations in the contiguous US. The issue obviously becomes one of whether the distribution of the classified stations is sufficiently even in those regions of the US that can have different warming trends.
Briefly described here, I used the GHCN (current version) monthly stations and RSU classifications for those stations as given by GHCN. I selected the period 1920-2005 in order to provide as many long series as I could. I used only series that started in 1920 and ended not earlier than 2005 and further had no more than 4 years intervening missing data. I regressed the trends for these stations versus Pop (RSU classification), latitude, longitude, altitude and proximity to water and the interaction of Pop with latitude, longitude, altitude and water.
I think it is true that a regression, that can be shown to be relatively free of clustering effects, gives more degrees of freedom to differentiate between population classification than doing something like using an urban station and nearest rural neighbors. This situation arises primarily because there are not that many urban stations and the station trends are very noisy. Obviously since the portion of urban stations is low any UHI effect on the overall US warming trend will be small. What is more of interest, for my preferences anyway, is how well the homogenizing algorithms react to an effect like UHI. To that end I looked at Adjusted and Unadjusted (TOB) GHCN series for the 1920-2005 period for maximum, minimum and mean temperature trends. The results are summarized in the five links listed below.
The results of my analysis as summarized and linked in the first two links below shows from the regressions that both the rural and suburban can have trends significantly lower than urban stations in some of the series and in the adjusted series. Also it interesting to note that the suburban stations tend to have the smallest trends of the three classifications but nearly equal to that of the rural stations.
The clustering of stations by classifications and trends are shown in the third and fourth linked maps below for the mean adjusted temperature series. The bottom table in the second link below shows a detailed analysis of the 6 zones for the contiguous US I selected with the trends and the portion of rural, suburban and urban stations all from the mean adjusted series. Overall these maps and tables do not show me any clustering effects that might significantly affect the overall averages and regression results, but I’ll leave that those who might want to view those results. Obviously there are clustering of trends and to a lesser extent RSU classifications, but the point here is whether it effects the overall results.
In the fifth link below I attempt to analyze the effects of altitude and proximity to large bodies of water for the mean adjusted series. Regressing trends against altitude indicates that trends increase with altitude but not significantly and the mean altitudes for the station classification increases in the order from lowest to highest from urban to suburban to rural.
Also in the fifth link is a table that shows the portion and total number of rural, suburban and urban stations close to large bodies of water and the trends for all stations close to large bodies of water and those not close to large bodies of water. Although the portion of station close to large bodies of water is small for all classification it is 3 times higher for urban stations (I guess as would be expected for knowing that large cities tend to grow large when near large bodies of water). Also the trends for stations close to large bodies of water are nearly 3 times greater those stations than not close. The mean trends for close to a large body of water stations for rural, suburban and urban are, respectively 0.222, 0.145 and 0.171. If we take the product of the portion close to water and the trend we get an idea of what the concentration of urban stations near the large bodies of water does to the overall trend by station classification and is the following: rural = 0.0067, suburban = 0.0043 and urban = 0.0154. Those differences between classifications are relatively small compared to the differences between the rural/suburban and urban stations.
*All trends are listed in the links below as degrees C per decade.
http://imageshack.us/a/img593/7664/rsutrendregressghcnmont.png
http://imageshack.us/a/img705/7333/meanadjunadjmonthlyghcn.png
http://imageshack.us/a/img839/9102/maptrendsghcnmonthlymea.png
http://imageshack.us/a/img577/8369/rsumapghcnmonthlymeanad.png
http://imageshack.us/a/img35/8792/proximitywateraltitudeg.png
@don montfort
speaking of the soap opera, I would say that not using the running averages of GAT and average atmospheric co2 are vital to the preservation of the soap opera, wouldn’t you?
It is quite hysterical that such simple questions send people into fits of rage. Bottom line is that the soap opera enthusiasts will not use the running averages, nor comment on them, because it wrecks the narrative. So instead, they treat the general public like suckers and keep saying the world has warmed using an extremely cherry picked starting point.
shazaam/owned.
Zeke, I suspect your long gone from this thread, but I looked at some cluster functions in R per what you said about suggestions from economists from Stanford. I used the cluster function clara from the library(cluster) in R to divide the CONUS into any number of regions which the function determined from the latitude, longitude and trends from the stations for the series GHCN monthly mean 1920-2005.
Obviously the problem one has when the rural, suburban and urban station are regressed together over the entire CONUS is that the result might be tainted by say a large portion of urban stations being located in warmer or colder region of the US. Other problems that occur if one attempts to compare means of trends for the R,S and U designations is that the numbers of urban station are small in number and the standard deviations for stations across the US can be large. In doing difference series with urban stations and nearest neighbors designated rural (and suburban) such as was the case in Zeke’s paper, I think the degrees of freedom get reduced substantially beyond what other methods might produce.
From these observations I was hoping that grouping the stations more or less objectively using a cluster function (although the number of groups remained my choice) would produce more uniform regions of warming and cooling, reduce the standard deviations and yet allow sufficient number of stations to better discriminate between R, S and U stations. After dividing the CONUS into 9 regions, I think I accomplished at least some of what I had hoped to be able to do. The results more or less agreed with what I found in regressing trends versus latitude, longitude, altitude, presence of large bodies of water and population classification rural, suburban and urban.
I need to do more work with groupings using cluster functions but I think it might be a step in the right direction.
kenneth how do you define close to a large body of water. and how large of a body of water
All temperature data and meta data for regressing station trends versus latitude, longitude, altitude , RSU population classification and proximity to water (and cluster analyses) were taken from the latest version of GHCN for CONUS and for covenience in taking a first look at RSU effects with newest GHCN version.