This post is an update on my earlier post on the same subject. In my earlier post I regressed linearly detrended Hadley global temperature anomalies since 1950 against a combination of low pass filtered volcanic forcing, solar forcing, and ENSO influence. For ENSO influence, I defined a low pass filtered function of the detrended Nino 3.4 index, which I called the Effective Nino Index (ENI).
My regression results suggested the rate of warming since 1997 has slowed considerably compared to the 1979 to 1996 period, contrary to the results for Foster & Rahmstorf (2011), which showed no change in the rate of warming since 1979.  There were several constructive (and some non-constructive) critiques of my post in comments made here at The Blackboard…. and elsewhere. This post will address some of those critiques, and will examine in some detail how the choices made by F&R generated “no change in linear warming rate” results; results which are in fact not those best supported by the data.
Limitations of the Regression Models
 It is important to understand what a regression, like done by F&R or in my earlier post, can and can’t do. If each of the coefficients which come from the regression are physically reasonable/plausible, then the quality of the regression fit indicates:
1) if the assumed form of the underlying secular trend is plausible, and
2) if there are important controlling variables not included in the regression
The difference between the regression model and the data (that is, the model ‘residuals’) is an indication of how well the regression results describe reality: Is the form of the assumed secular trend plausible? Do the variables included in the regression plausibly control what happens in the real world?  However, if the coefficients from the regression output are not physically reasonable/plausible, then even a very good “fit” of the model to the data does not confirm that the shape of the assumed secular trend matches reality, and the regression results may have little or no meaning.
Some Substantive Critiques from My Last Post
The fundamental problem with trying to quantify and remove the influences of solar cycle, volcanic eruptions, and ENSO from the temperature record was pointed out by Paul_K, who noted that selection of a detrending function, which represents the influence of an unknown secular trend, is essentially circular logic. The analyst assumes the form of the chosen secular function (how the secular function varies with time: linear, quadratic, sinusoidal, exponential, cubic, etc, or some combination) in fact represents the ‘true’ form of the secular trend. The regression done using a pre-selected secular function form is then nothing more than finding the best combination of weightings of variables in the regression model which will confirm the form of the assumed secular trend is correct.
Hence, any conclusion that the regression results have “verified” the true form of an underlying trend is a bit circular… you can’t verify the shape of an underlying trend, you can only use the regression to evaluate if what you have assumed is a reasonable proxy for the true form of the secular trend. In the case of F&R, the assumed shape of the secular trend was linear from 1979; in my post the assumed secular trend was linear from 1950. Both suffer from the same circular logic.  F&R also allow both lag and sensitivity to radiative forcing to vary independently, which allowed their regression to specify non-physical lags and potentially non-physical responses to forcings, which together lead to the near perfect ‘confirmation’ of their assumed linear trend.  All of the regressions in this post, as well as in my original post, require that both solar and volcanic forcings to use the same lag, though that lag is free to assume whatever value gives the best regression fit, even if the resulting lag appears physically implausible.
Nick Stokes suggested substituting a quadratic function (with the quadratic function parameters determined by the regression itself) and went on himself to compare the regression results for linear and quadratic functions for 1950 to 2012 and 1979 to 2012.  Like me, Nick used a single lag for both solar and volcanic influences. Nick concluded that with a quadratic secular function, there is some (not a lot) deviation from a linear trend post 1979, which varies depending on what temperature record is used. Nick’s results are doubtful because simply choosing a quadratic secular function is just as circular as choosing a linear function.  Some of the lag constants Nick’s regression found for the 1975 to 2012 period (eg. ~0.11) appear physically implausible (much too fast).
Tamino (AKA Grant Foster of F&R) made a constructive comment at his blog: a single lag constant for solar and volcanic influences (a “one box lag model”) was not the best representation of how the Earth is expected to react to rapid changes in forcing like those due to volcanoes, and that a two-box lag model with a much faster response to account for rapid warming of the land and atmosphere would be more realistic. I have included this suggestion in my regressions.
Commenter Sky claimed that basing ENI on tropical temperature responses was “a foolishness” (I strongly disagreed) but his comments prompted me to look for any significant correlation between the ENI and non-tropical temperatures at different lags, and I found that there is a very modest but statistically significant influence of the ENI of non-tropical temperatures at 7 months lag. Incorporating both ENI and 7-month lagged ENI slightly improves the regression fit in all cases I looked at, and generates an estimated global response for ENI (not lagged 7 months) which is close to the expected value of half the response for the tropics.  (I will describe the (modest) modifications I made based on Tamino’s suggestion and on a 7-month lagged ENI contribution in a postscript to this post.)
Finally, Paul_K suggested that a way to avoid logical circularity was to try a series of polynomials in time, of increasing order, to describe the secular trend (with time=0 at the start of the regression model period) and with the polynomial constants determined by the regression itself. The resulting regression fits can then be comparing using a rational method like AIC (Akaike Information Criterion) to determine the best choice for order for the polynomial (the minimum AIC value is the most parsimonious/best). For a linear regression with n data points and M independent parameters, the AIC is given approximately by:
AIC = n * Ln(RSS) + 2*M
Where Ln is the natural log function and RSS is the Residual Sum of Squares from the regression (sometimes also called the “Error Sum of Squares”). M includes each of the variables: solar, volcanic, ENI, Lagged ENI, secular function constants, and the constant (or offset) value. Higher order polynomials should allow a better/more accurate description of secular trends of any nonlinear shape, but each added power in the polynomial increases the value of M, so a better fit (reduced RSS) is ‘penalized’ by and increase in M.  A modified AIC function, which accounts for a limited number of data points (called the AICc) is better when the ratio of n/M is less than ~40, but this ratio was always >40 for the regressions done here.
AIC Scores for Polynomials of Different Order
The ‘best’ polynomial to use to describe the secular trend, based on the AIC, depends as well on whether or not you believe that the influences of volcanic forcing and solar forcing are fundamentally different on a watt/M^2 basis. That is, if you believe that solar and volcanic forcings are ‘fungible’, then those forcings can be combined and the regression run on the combined forcing rather than the individual forcings.  In this case, the best fit post 1975 is quadratic. Troy Masters (Troy’s Scratchpad blog, based on a suggestion from a commenter called Kevin C) has showed that summing the two forcings improves a regression model’s ability to detect a known change in the slope of secular warming in synthetic data.
If a regression is done starting in 1950 (as in my original post) with solar and volcanic forcings treated separately, then it appears the best, or at least most plausible polynomial secular trend is 4th order, which represents a ‘local minimum’ in AIC… lower than 5th order. AIC scores of 6th order and above are smaller than 4th order,  but the regression constants for solar and volcanic influences do not “converge” on similar values for each higher order polynomial; they instead begin to oscillate, indicating that the higher order terms (which can simulate higher frequency variations) are beginning to ‘fit’ the influences of volcanoes and solar cycle, rather than a secular trend. In any case, using  a 4th order polynomial for the regression starting in 1950 generates a much improved fit compared to an assumed linear secular trend.
1975 to 2012
Figure 1 shows the AIC scores and lag constants for regressions from 1975 to 2012.
Â
The minimum AIC score is for a third order polynomial. The corresponding regression coefficients (with +/- 2-sigma uncertainties) are shown below:
Volcanic                              0.14551 +/-0.0248
Solar Cycle                         0.47572 +/-0.2034
ENIÂ Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 0.08253 +/-0.0143
7 Mo Lagged ENIÂ Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 0.03179 +/-0.0144
Linear Contribution            0.01335 +/-0.00897
Quadratic Contribution     0.0003753 +/- 0.000542
Cubic Contribution             (-8.655 +/-9.18)*10^(-6)
Constant                               0.02384 +/-0.0383
R^2Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 0.828
F Statistic                              306
Figure 2 shows the regression model overlaid with the secular component of the model. The secular component is what is described by the above linear, quadratic, cubic contributions plus the regression constant. Figure 3 shows the regression model overlaid with Hadley temperature anomalies.   The model matches the data quite well. The residuals (Hadley minus model) are shown in Figure 4. The residuals are reasonably uniform around zero, and show no obvious trends over time.



Figure 5 shows the individual contributions (ENSO, solar cycle, volcanic) along with their total.

Figure 6 shows the original and ‘adjusted’ Hadley data, where the influences for ENSO, solar cycle, and volcanoes have been subtracted form the original Hadley data. I have included calculated slopes for 1979 to 1997 (inclusive) and 1998 to 2012 (inclusive). The best (most probable) estimated trend for 1979 to 1997 is 0.0160 C/yr, while from 1998 to 2012 the best estimate for the trend is 0.0104C/yr, corresponding to a modest (35%) reduction in the rate of recent warming. (edit: 0.0160 should have been 0.0171, 0.014 should have been 0.0108; the reduction is 37%)

1950 to 2012
Figure 7 shows the AIC scores and lag constants for regressions from 1970 to 2012.

The local minimum (best) AIC score is for a fourth order polynomial.  At orders 6 and above the AIC score continues to fall, but without convergence of solar and volcanic coefficients, which suggests to me that the higher order polynomials are beginning to interact excessively with the (higher frequency) non-secular variables we are trying to model, and the continued fall in AIC score is not indicative of a true improvement in accuracy of the higher order polynomials as a secular trend.  I adopted the fourth order polynomial as the most credible representation of the secular trend. The corresponding regression coefficients (with +/- 2-sigma uncertainties) are shown below:
Volcanic                               0.1346 +/-0.0234
Solar Cycle                         0.3283 +/-0.170
ENIÂ Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 0.0972 +/-0.0122
7 Mo Lagged ENIÂ Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 0.0245 +/-0.0121
Linear Contribution            0.00804 +/-0.00828
Quadratic Contribution      -0.000772 +/- 0.000542
Cubic Contribution             (2.97 +/-1.32)*10^(-5)
Quartic Contribution          (-2.7 +/-1.06)*10^(-7)
Constant                               -0.0137+/-0.0374
R^2Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â 0.83691
F Statistic                              473
Figure 8 shows the regression model overlaid with the secular component of the model. The secular component is what is described by the above linear, quadratic, cubic, and quartic contributions plus the regression constant. Figure 9 shows the regression model overlaid with Hadley temperature anomalies.  The model matches the data quite well. The residuals (Hadley minus model) are shown in Figure 10.



Figure 11 shows the original and adjusted Hadley data, where the influences for ENSO, solar cycle, and volcanoes have been subtracted form the original Hadley data. I have included calculated slopes for 1979 to 1997 (inclusive) and 1998 to 2012 (inclusive). The best estimate for the trend from 1979 to 1997 is 0.0164 C/yr, while from 1998 to 2012 the best estimate for the trend is 0.0084C/yr, corresponding to a 49% reduction in recent warming.

But what if radiation is fungible?
The divergence between the regression diagnosed ‘sensitivity’ to changes in solar intensity and volcanic aerosols is both surprising and puzzling.   The divergence is reported (albeit to a smaller extent) in the 1950 to 2012 regressions as well as the 1975 to 2012 regressions. In each case, the regression reports a best estimate response to solar cycle forcing which is more than twice as high as volcanic response on a watts/M^2 basis.  Lots of people expect the solar cycle to contribute to total forcing in a normal (fungible) way. Figure 12 (from GISS) shows that for climate modeling, the folks at GISS think there is nothing special about solar cycle driven forcing.

For the diagnosed divergence between solar and volcanic sensitivities to be correct, there must be an additional mechanism by which the solar cycle substantially influences Earth’s temperatures, beyond the measured change in solar intensity. I think convincing evidence of such a mechanism (changes in clouds from cosmic rays, for example) is lacking, although I am open to be shown otherwise.   But if we assume for a moment that there really is no significant difference in the response of Earth to solar and other radiative forcings, which seems to me plausible, then the above regression models ought to be modified by combining solar and volcanic into a single radiative forcing.
When the regressions are repeated for 1975 to 2012 with a single combined forcing (the sum of individual solar and volcanic) the minimum AIC score is for a quadratic secular function (rather than cubic when the two are independent variables), but the big change is that the regression diagnoses a longer lag for radiative forcing and a stronger response to radiative forcing (which is of course dominated by volcanic forcing). Figure 13 shows the AIC scores and lag constants for polynomials of different orders when solar and volcanic forcings are combined.  The minimum AIC score with the combined forcings (quadratic secular function, 649.2) is slightly higher than the minimum for separate forcings (cubic secular function, 648.2), which lends some support to higher sensitivity for solar forcing.

Figure 14 shows the regression model and Hadley data, and Figure 15 shows the Hadley data adjusted for ENSO and combined solar and volcanic forcing.


The 1998 to 2012 slope is 0.0072 C/yr, while the 1979 to 1997 slope is 0.0165 C/yr; the recent trend is 44% of the earlier trend.
Why did F&R get different results?
The following appear to be the principle issues:
1. Allowing volcanic and solar lags to vary independently of the others.
2. Accepting physically implausible lags.
Treating solar and volcanic forcing as independent, combined with number 1 above seems to have some unexpected consequences. Figure 16 shows the lagged and un-lagged volcanic forcing along with the un-lagged solar forcing. The two major volcanic eruptions between1979 and 2012 (El Chichon and Pinatubo) happen to occur shortly after the peak of a solar cycle.

The solar and volcanic signals are partially ‘aliased’ by this coincidence (that is, both acting in the same direction at about the same time), while the decline in solar intensity following the solar cycle peak in ~2001 did not coincide with a volcano. Since there was a considerable drop in rate of warming starting at about same time as the most recent solar peak, and since the regression can “convert” some of the cooling that was due to volcanoes into exaggerated solar cooling due to aliasing, the drop in the rate of warming after ~2000 can be ‘explained’ by the declining solar cycle after ~2001. In other words, aliasing of solar and volcanic cooling in the early part of the 1975-2012 period, combined with free adjustment of ‘sensitivity’ to the two forcings independently, gives the regression the flexibility to increase sensitivity to the solar cycle by reducing the sensitivity to volcanoes, so that the best fit to an assumed linear secular trend corresponds to a larger solar coefficient. Allowing the solar and volcanic forcing to act with different lags further increases the ability of the regression to increase solar influence and diminish volcanic influence. All of which contributes to the F&R conclusion of “no change in underlying warming trend.”
Of course, the same aliasing applies to the regression for 1950 to 2012, but since there are more solar cycles and more volcanoes in the longer analysis, and those do not alias each other well, the regressions for the longer period reports a smaller difference in ‘sensitivity’ to solar and volcanic forcings. Â For example, with an assumed linear secular trend (similar to F&R, but using one lag for both solar and volcanic forcings), the 1975 to 2012 regression coefficients are: volcanic = 0.1294, solar = 0.5445, while the best fit regression from 1950 (4th order polynomial secular function) the coefficients are: volcanic: 0.1346, solar = 0.3283.
It will be interesting to see how global average temperature evolves over the next 6-7 years as the current solar cycle passes its peak and declines to a minimum. If F&R are correct about the large, lag-free influence of the solar cycle, this should be evident in the temperature record…. unless a major volcano happens to erupt in the next few years!
Conclusions
There is ample evidence that once natural variation from ENSO, solar cycles and volcanic eruptions is reasonably accounted for, the underlying ‘secular trend’ in global average temperatures remains positive.   But it is also clear that the best estimate of that secular trend shows a considerable decline in warming compared to the 1979 to 1997 period. The cause for that decline in the rate of underlying warming is not known, but factors other than ENSO, volcanic eruptions, and the solar cycle are almost certainly responsible.
Postscript
In my last post I showed how the low pass filtered Nino 3.4 index correlates very strongly with the average temperature between 30N and 30S, and can account for ~75% of the total tropical temperature variation.  To check for the possible influence of ENSO (ENI) on temperatures outside the tropics, I first calculated the “non-tropic history” from the Hadley global history and Hadley tropical history (non-tropics = 2* global – tropics). I then checked for correlation between the non-tropics history and the ENI (which is calculated from the low pass filtered Nino 3.4 index) at each monthly lag from 0 to 12 months. Significant correlation is present starting at ~4 months lag through ~11 months lag, with the maximum correlation at 7 months lag from the ENI. I then incorporated this 7-month lagged ENI as a separate variable in the regressions discussed above, and found very statistically significant correlation in all regressions. The coefficient was remarkably consistent in all regressions, independent of the assumed secular trend function, at ~1/3 that of the global influence of ENI itself. (The ENI coefficient itself was also remarkably consistent.) The 7-month lagged ENI influence was significant at >99.9% confidence in all regressions. The increased variable count was included in the calculation of AIC.
I used a simple dual-lag response model (two-box model) instead of a single lag response because land areas (and atmosphere) have much less heat capacity than ocean, and so react much more quickly to applied forcing than the ocean. The reaction of land temperatures would be expected to be in the range of an order of magnitude faster based on relative (short term) heat capacities, which on a monthly basis makes the land response essentially immediate (lag less than a few months).   If land and ocean areas were thermally isolated, we would expect the very fast response of land to represent ~30% of the total, and the slower response of the ocean to represent ~70%. That is, in proportion to the relative surface areas. However, land and ocean are not thermally isolated, and the fast land response is reduced by the slower ocean response because heat exchanges between land and ocean pretty quickly. Modeling this interaction would appear to be a non-trivial task, but I guessed that a simple approximation is to reduce the relative weightings of land and increase that of the ocean, and assumed 15% ‘immediate’ response and 85% lagged response. The lag constants optimized in the regression applied to only 85% of the forcing; 15% of the forcing was considered essentially immediate. The above appears to improve R^2 values a bit in nearly all regressions I tried, but does not impact the conclusions: the underlying secular trend appears lower from 1998 to 2012 than from 1979 to 1997.