Bart Verheggenhas posted a response to Brandon’s post. Bart writes:
It should not come as a surprise that many papers expressed no position on the causes of global warming: Many papers matching the search terms “global warming†or “global climate change†don’t deal with attribution of the warming, but with impacts (48% of the total) or mitigation (28% of the total) of climate change (e.g. the change in the yield of corn as a function of temperature).
Certainly, papers on impacts, do not necessarily discuss attribution. They are not about attribution. But this is hardly a good reason to include those that happen to mention that motivation of their work in the tally for consensus while zeroing out those that fail to mention the motivation of their work. Doing so introduces an odd bias because it is extremely rare for anyone to list the “things that did not motivate this work” in an abstract. Consequently, the tendencies when writing abstract would be
- The fraction who endorse AGW as true will tend to mention global warning as a motivation
- The fraction who obtained funding from an agency that funds impact or mitigation work that is necessary if AGW is true will tend to mention global warming as a motivation. They will tend to endorse (so the head of their program can use that to support the program.)
- The fraction whose work on topics like ecology — which have nothing to do with attribution– and who happen to not endorse AGW and/or whose work is not funded by AGW will simply not mention it.
I’d like to explain this a bit further by taking a snippet from a comment in your Bart’s comment thread. Writing to support Bart Verheggen’s advocacy for subtracting out all the “4” (neutral) ratings when computing the 97% of scientists believe in global warming Bob Brand says
Therefore the number of category 4 papers is irrelevant to the degree of consensus – another 1000 papers on biofuels does not in any way dilute OR strengthen the consensus!
I agree on the final bit which I italicized: That is papers on biofuels do not tell us anything about the consensus. The difficulty is this is a poor argument for a method used in the SkS paper, and which Bart endorses. That method computes the fraction of support by:
- Using search terms that first include papers on biofuels.
- Rates them all.
- Keeps those that with ratings that are not equal to 4, while throwing out those that are rated 4.
If one believes that papers that discuss biofuels tell us nothing about consensus, then one should remove all papers whose focus is the development of biofuels.
Moreover, the full text of Bob Brand’s comment seems to suggest the biofuels papers were rated category 4 or at least that they have not been assigned to categories 1-3. This is contrary is true.
One can do a search on category 3 papers that contain the word “a” included in the computation; that gets you all the papers in category 3.
On the first page you will find:
Fuel Ethanol From Cellulosic Biomass
Authors: Lynd, Lr; Cushman, Jh; Nichols, Rj; Wyman, Ce (1991)
Journal: Science
Category: Mitigation
Endorsement Level: 3. Implicitly endorses AGW without minimising it
If you really believe papers on biofuels should neither dilute nor strengthen the consensus, this paper rated “3” to the consensus should be ignored because it is about biofuels. They should just be ignored when someone gave them a 4 but included when someone gave them some other number.
Inspired by finding a paper on ethanol used to jack up the “pro AGW” consensus fraction, one can do a search on ethanol. We get 177 papers. We can do subsearches for each rating. It turns out if exclude the “4s”, the outcome is 99% endorse global warming; this is higher than the 97% found for the “mean”. (That’s 102 out of 103 papers. Including 4’s there are 177 papers.) These means that including these papers which are utterly irrelevant to attribution tends to push the estimate of consensus for agreement with AGW upward. How much? We don’t know.
If you do the search on the term Bob Brand mentioned biofuels, using the SkS method 60/60 papers “endorse” so you get 100% of papers endorse. This is also higher than 97%.
The reason these papers push the apparent pro-AGW consensus up is obvious:
Global warming is a motivation for the government program that funds these projects. Motivation is frequently mentioned in abstracts. While there may be some unknown number working in fuels programs, and possibly even biofuels programs who reject AGW, for those people AGW falls in the list of “non-motivations”. It is quite rare for authors of peer reviewed papers to include a list of things of “non-motivations” in their abstracts or papers as the list of non-motivations is irrelevant to their work.
The consequence of this asymmetry in mentioning motivations while failing to mention non-motivations is that including papers on topics that are irrelevant to attribution will tend to drive the appearance of endorsement to artificially high levels. One might even see 100% endorsement even if the consensus for AGW were low. With respect to assessing endorsement of AGW of authors writing about biofuels: the SKS method can’t even us the ratio of authors of papers in “biofuels” who endorse global warming to those whose authors don’t. (Mind you: I would guess it’s quite high. But my guess isn’t based on the SkS result. )
As for the justification of the methodology of Cooks survey: if someone wants to limit the tally to the fraction to papers doing attribution my suggest on is limit the tally to papers that do attribution directly. That is drop out both the “= 4” and the “≠4 papers” that do not fit “attribution”. Suggesting that one should drop out the “=4” because some papers aren’t about attribution while retaining skads of papers that are not about attribution is unbalanced.
After culling to remove papers that are not about attribution, what should be done with the “=4”? The simplest thing is to recognize that if a paper that is about attribution gives a neutral result, there is no reason to drop it from consideration. It merely means that a paper that is specifically about attribution actually resulted in a “undecided” or “neutral” result.
The fact that a paper on attribution resulted in “undecided” or “neutral” is of interest if one is trying to gauge the consensus on either the fact of warming or the cause of warming.
I’d go further. If someone wants wishes to take a short cut and limit to papers that are WGI topics (the area that touches on attribution), limit the group of papers to those on WGI. Drop out all the WGII, WGIII topics and the one on feminism. (Does that fit into any WG?) In this case, one might be able to argue over the dropping or keeping of the “4s” but at least we are getting closer to fair because WGI topics at least might hypothetically discuss attribution, though even then some will not. This is imperfect, but it is vastly superior to including papers that are not even about climate in the pool.
(Caveat: I am dubious about the whole idea of doing this using abstracts pulled in using these two word searches. It seems to miss at least some papers on “trends” and “attribution”. But if it is going to be done, the results shouldn’t be based on abstracts of papers with titles like “Temporal Variation In Growth-rate And Age At Maturity Of Male Painted Turtles, Chrysemys-picta”.)
What could someone do guesstimate the effect of excluding totally irrelevant papers?
I’m not sure how a recalculation could be performed in a robust way. Ideally, the entire project should create a database using search terms that are confirmed to pick out the papers on attribution or WGI topics.
However, one might try to do an order of magnitude estimate by performing a recalculation by removing “mitigation”, “impact” and “not climate related papers from the list of ratings; then recompute the consensus using both the “keep =4” method and “toss out =4” method. I suspect the numbers will change a rather small amount–my swag is support for consensus for AGW computed including “4”s will rise from the low 1/3rd in the study while the support for consensus computed by removing the 4’s will drop. I’m guessing we might get 50% for the ‘keep fours’ method and 90% for the drop fours method. I’m pulling those numbers out of my tuckus.
While I’ve pulled number out of my tuckus, I do expect the ‘keep fours’ consensus and the ‘ignore 4s’ computed consensus estimates will be closer in value to each other in part because eliminating papers on “feminism”, “biofuels” and “Sex-ratio Of Hatchling Loggerhead Sea-turtles – Data And Estimates From A 5-year Study” results in set of papers that contains fewer whose contents are totally irrelevant to assessing a consensus on whether AGW is real.
If someone can point me to a scraped set (I know it’s mentioned in comments somewhere) I’ll be happy to tally up for the quick estimate.
Reminds me a bit of the “nano” hype a few years back.
Virtually every paper about “nano technology” either implicitly or explicitely “endorsed” the field. (Why would anyone say “Here we report on our work in nanotechnology, which everyone knows is bogus”?).
Government funding is a powerful force.
John is correct, it is all about following the money and the trail of corruption and deceit it leaves behind.
THe GW’ers, for years, have vilified skeptics by accusing them of being in the pocket of big oil, all the while taking multimillions from big oil themselves to help the oil companies stifle competition.
But that ignores the braying jackasses and trumpeting elephants in the barnyard. Our government exists to solve problems by taking taxpayers money and regulating them. What better problem to solve than the weather? That is why the problem evolved from warming to change. The climate change meme is unfalsifiable and unfixable, just like the wars on poverty, drugs, etc. etc.
The only studies worth the paper they are printed on are self financed. All the rest of the papers qualify as poor quality toilet paper.
A puzzlement is why so much effort in sceptical blogospheres is being devoted to a flawed and obviously biased paper. It accomplished its PR objective, true. That has nothing to do with either science or truth. Nothing said here or on other blogs is going to change that PR fact, although you may feel better after proving the ‘cooked’ books to yourselves.
You want to be effective, resort to Cook tactics against Cook and his. This is about politics, not about the subtleties of auto regressive residual correlations.
Cook won a round. By cheating. Now how do you all win the next ten rounds without resorting to his cheats?
Good news is, nature is on your side what with the missing heat still missing despite Trenberth’s assertion to the contrary (not yet deconstructed here or elsewhere) and with that little awkward detail of the pause, which the other side said in peer reviewed literature would about now falsify their models.
Why don’t we get less cerebral and more effective in stopping this nonsense?
Caveat: if you read my ebooks, you might begin to realize there are collateral, much bigger, more certain problems which do not admit the same laissez faire solutions as to CAGW. But, it would be a step forward to debate real science issues rather than faux CAGW.
Rud writes “Cook won a round. By cheating. Now how do you all win the next ten rounds without resorting to his cheats?”
On the whole skeptics are skeptical of results and delight in uncovering the biases and inconsistencies in scientist’s arguments. Skeptics, on the whole, dont go out to do “alternative science”. Thats not really what a skeptic is all about IMO.
The Cook paper is perfect fodder for examining both the bias and inconsistency in argument.
From the paper “Explicit endorsements were divided into non-quantiï¬ed (e.g., humans are contributing to global warming without quantifying the contribution) and quantiï¬ed (e.g., humans are contributing more than 50% of global warming,”
It seems to me that the definition of AGW as is being defended by SkS and this paper is actually that the contribution to observed warming is greater than 50%.
Anything less and an abstract rejects AGW as defined by the IPCC or more specifically “most of the global warming since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations”
Hence non-quantified “endorsements” are actually equally likely to be rejections, not just “ignorable”.
TimTheToolMan
No. An implicit endorsement in the abstract (“3”) neither rejects nor endorses AGW as defined by the IPCC (1).
If this was an ordinary survey where a person was asked to pick which of the answers best fit their views, then their picking “3” would represent not picking “5”.
But this isn’t a normal survey– authors merely wrote an abstract– sometimes as far back as the 90s– and when a different person evaluated the abstract, that person evaluated it to correspond to “3”. Moreover, when the authors were writing the abstracts they didn’t know that someone would read them to answer that question. That makes interpreting the meaning of most the answers more difficult than for most questions.
Provided the “implicit endorsement” rating is accurate, an implicit endorsement could mean anything from “AGW is true (I’m not going to reveal how strong I think it is.)” to “AGW is ±50% or warming”.
The real difficulty is that notwithstanding the fact that the definitions in the papers say that an abstract saying warming is <50% from AGW means "rejection", some abstracts that on reading look like “implicit endorsement” and even some “explicit endorsement but not quantified”, might actually be “rejections”. Because when things are implicit and the person doesn’t quantify, and they don’t happen to be mentioning attribution, you cannot know whether they think the warming is >50% AGW or < 50%. Given the huge number of “implicit endorsements” we can’t know whether some fraction of those might not correspond to the category the SkS raters considered “reject”.
I’ll go fish out some examples.
TimTheToolMan, you’re discussing the same issue I’ve been covering for a couple days now. The “consensus” Cook et al’s results show merely says humans cause some amount of warming, a meaningless result. If we look at any issues that actually matter, like how much warming humans are responsible or what the effects of the warming will be, Cook et al’s results say there is no consensus.
The people responsible for the study have chosen to grossly misrepresent their results by taking their results which say, “Humans cause some global warming” as saying, “Humans are the primary cause of global warming.” It is a pure fabrication, and it is necessary because their data does not give the answer they wanted.
And people are jumping to defend it.
Here’s the first from “mitigation” if you search for “implicit endorsement” papers that contain the letter “a”
This is “implicit endorsement”. (I wouldn’t rate it as such but the SkS rating gave this “implicit endorsement”.
Now let’s look at it from rating the abstract 1-7 on the SkS scale:
1) It does mention global warming.
2) It does qualitatively assess what problems could be intensified by global warming if global warming is true and has some uncertain magnitude.
The difficulty is that I read (2) as “If X true, then these are the things that could happen” construction. That is: a counter factual. The person writing it does not necessarily have an position on the truth of X”. The aren’t necessarily endorsing X– nor are they disendorsing X.
And moreover, in this particular case, he says that X is not well understood, so it really truly does look like a “counter factual” type construction. GHG’s aren’t mentioned (just “chemicals”. I don’t think CO2 or methane are normally called “chemicals”, though the could be.)
Technically, we don’t have any notion about the magnitude nor the cause of the warming assumed in this study. Likely the cause of warming doesn’t matter at all to the study. It will simply be “if it warms (for any reason) the public health consequence are.” If we merely go by reading this paper could equally well correspond to less than 1/2 observed warming is man made– you just don’t know. They don’t say. And why should they be specific? It doesn’t matter for a public health study.
lucia:
I assumed he meant “fails to support” rather than “rejects” because his next sentence says those papers “are actually equally likely to be rejections.” I took that to mean he believes the abstracts that endores AGW without quantifying it are “equally likely” to reject or endorse the IPCC position. Basically the same thing you’re saying.
If I’m right, he just misspoke in one sentence. Hopefully he’ll clarify.
Lowering the statistical significance and declare victory is an old and honorable procedure 🙂
On the DSM “bible” for Psychotherapy
The DSM-IV, which has earned $100 million, keeps the organization in the black. Faced with a looming deadline and terrible data, Mr. Greenberg suggests, the DSM directors did what any reasonable, self-protecting institution would do: They lowered the statistical criteria for acceptable standards of reliability and turned defeat into victory. As Allen Frances puts it in “Saving Normal,” they accepted agreements among raters that were “sometimes barely better than two monkeys throwing darts at a diagnostic board.”
Lucia:
Here is my scraped data set… but are you looking for something that has the abstracts as well as their ranking?
Prof lewandowsky is back blogging. New shaping tomorrows world blog post, supporting the TCP project and John Cook
Brandon writes “I took that to mean he believes the abstracts that endores AGW without quantifying it are “equally likely†to reject or endorse the IPCC position.”
Yes. This is the position I was trying to take. For example if an author believes 10% of the warming is caused by anthropogenic CO2 then that is a fairly clear rejection of the IPCC position however if that author believes anything up to half might be due to the anthropogenic CO2 to then is also a rejection but a less strong one.
Either way, both cases do not count that author as supporting the consensus on AGW except for a definition of AGW that is much broader than the IPCC definition (and IMO has little value)
I’d be happily wiling to agree that 97%+ scientists would agree that CO2 has a measurable warming role in the atmosphere. But thats miles away from the IPCC position.
Lucia writes “Technically, we don’t have any notion about the magnitude nor the cause of the warming assumed in this study.”
Yes, but that is precisely what Cook and co are tying to establish. AGW isn’t AGW unless it conforms to the IPCC definition. If they choose to use this abstract then they must take the consequences of that choice and in this case the author is unwilling to agree about the magnitude
“Because the atmospheric effects of ozone depletion are fairly well characterized, quantitative risk estimates have been developed. However, because the atmospheric effects of global warming are less understood”
…atmospheric effects of CO2 being less understood in the context of the well understood O3 effects implies a non-endorsement of the IPCC position of CO2 causing more than half the warming. If they were more sure on the magnitude (being in line with the “consensus”) then they wouldn’t have expressed their doubts.
So IMO right there is an author who could easily be a scientist who rejects AGW as defined by the IPCC.
Carrick (Comment #113433)
May 19th, 2013 at 12:39 pm Edit This
I was looking for “by category” i.e. mitigation etc. I manually got a few I wanted and commented a bit.
It would be nice to get this some way other than scraping.
Slightly offtopic question: isn’t AGW the consensus of 97% of “skeptics”? Don’t most visitors to sites like Watts’s believe that mankind has increased CO2, a greenhouse gas, and thus caused global ground level temperatures to go up?
I ask because it seems the first problem is the “strawman fallacy” of creating a caricature of your opponents position, that they deny the basic evidence. Certainly some skeptics have that position, but it seems the majority don’t. The debate isn’t whether AGW is true, but things like sensitivity and consequences of warming.
I don’t know what the percentage is. I don’t think the numerical value is knowable with any certainty. (It’s theoretically measureable. But I don’t think there are any good methods for doing it.)
I would say most– in fact the overwhelming majority of– people who are called “skeptics” or self identify as skeptics think CO2 has increased, is a greenhouse gas and causes temperature to go up. Yes.
I do think that Monckton recently has a head on with O’Sullivan one of the “SkyDragons”. (See http://www.amazon.com/Slaying-Sky-Dragon-Greenhouse-ebook/dp/B004DNWJN6) Monckton was on the “CO2 is a greenhouse gas side. And Monckton was posting at Watts site. Spencer who also believes CO2 does something also posted at Watts. So Watts would be in the “does not deny radiative physics” camp and also the “does not deny we’ve increased CO2” camp. I don’t know what magnitude of effect Watts advocates– but it’s not 0.
I think the “SkyDragons” think CO2 doesn’t do anything. So there are some people who think CO2 does nothing.But Watts and Monckton have openly criticized “The SkyDragons” for their denial of radiative physics.
Yes. And that’s where the “spin” aspects of the SkS/TCP paper come into play.
Here is an abstract that was rated as neutral:
“The natural greenhouse effect of the Earth is strongly influenced by the radiative effects of water vapour and clouds in the atmosphere, which control the energy absorbed from the sun, and that lost through thermal emission to space. Any perturbations to the climate balance, for example through so-called ‘radiative forcing’ due to increasing CO2 amounts, variations in solar constant, or other causes, can be amplified by the feedback processes that involve water in its various phases. The radiative cooling of the Earth in the absence of clouds has recently been shown to be dominated by emission from upper-tropospheric water vapour, in the far infrared portion of the spectrum, and this is illustrated: observations of this radiative flux, and of the distribution of water vapour in the upper troposphere, are urgently needed. The role of clouds is discussed, and it is noted that their response to global warming is not presently unambiguously determined with available models, due to the complexity of competing processes: again, as in the cloud-free case, more accurate global observations are needed. The paper is illustrated by data from satellite experiments, most notably the Earth Radiation Budget Experiment sponsored by NASA.”
According to the consensus on this site, it should have been rated as supporting the consensus on AGW because it acknowledges that there is such a thing as CO2 forcing. Based on the instructions for rating, however, this is clearly (at most) a neutral paper because it gives no indication that anthropogenic forcings are or have been stronger than natural forcings in recent times. I would have been much more inclined towards rating it at 5 (implicit rejection) rather than 3 (implicit acceptance).
I know you all want to fit this survey into your old narratives as to why consensus surveys must be rejected. I did not expect (on this site at least) the extent to which you are prepared to sweep evidence under the carpet to do so.
Tom Curtis
I never said this one “would have been” rated that 3. I think the ones that are actually on topics related to attribution tend to be easier to rate on your scale. (In fact, that’s the topic of the main post on this page).
The reason atrribution papers– like the example you fished out– fit your rating system fairly well is that to a large extent papers touching on attribution are explicit and quantify global warming. The one you picked happens to be explicit and more or less says “we don’t know consequence”. Actually saying “don’t know” is difficult to misrate. That’s why this particular post is suggesting that it would be best to limit the rating to papers touching on attribution.
The largest category of potential problem ratings are those where the endorsement of AGW is implicit with no quantification, and the topic is unrelated to attribution (e.g. public health, biofuels etc.) For details on the ones that I would say have a strong potential for being misrated, see http://rankexploits.com/musings/2013/on-the-consensus/#comment-113480 responding to your other comment on the other thread.