Very stupidly, I wrote a whole post on an NH ice extent bet without noticing I’d read in December volume data instead of September extent minimum data. This is the rewrite. Without the stupid blunder, Rob has a pretty good chance of winning and an even better chance of not losing.
I wasn’t planning on blogging about the Connolley-Dekker Ice Bet. But, the topic came up in comments, and Rob Dekker wrote:
Lucia, regarding the Connolley-Dekker (yep, that’s me) bet, you write :
RD’s bet is well outside my current 95% uncertainty intervals for prediction. Connolley is well inside the uncertainty intervals.
I’m curious, did you already present the model that you used that defined these statements, and if not, can you please present it ?
Answers: I had not explained my method for estimating who was going to win that bet, and I’m not even sure my current method will match whatever method I was using at the time I wrote that. I can present the current model for predicting who will win the Connolley-Dekker bet. RD is unlikely to lose. (Why did I say he was in comments earlier? I don’t know. I know it wasn’t the same mistake as the whole post because I hadn’t yet downloaded PIOMAS data at the time I wrote the comment. I may have mis-read the bet thinking it was for 2011. Anyway, this should explain to Rob how I do try to estimate odds based on extrapolating data when I actually do it. )
Weighted model
As some are aware, I have taken to using a “weighted model” approach to obtaining the best estimate for a future outcome based on past data. The general method is discussed here. For this post, I am doing a “tweak” which is to ignore deleting models based on lack of statistical significance of the fitting parameters.
Partial application of weighted prediction method to Connolley-Dekker Bet
The Connolley-Dekker Bet is described here:
If both NSIDC and IARC-JAXA September 2016 monthly average sea ice extent report are above 4.80 million km^2, RD pays WMC US$ 10,000. If both are below 3.10 million km^2, WMC pays RD US$ 10,000. In all other cases the bet is null and void
Note it is based on IARC-JAXA and NSIDC. To shorten discussion, I’m going to present analysis of NSIDC only (particularly as I haven’t done it for JAXA.) Based on the statement of the bet, my understanding is that RD will pay WMC if the NSIDC and JAXA Sept. extent average fall above the upper grey dashed horizontal like in 2016. WMC will pay RD if lower grey dashed horizontal in 2016. If the extent falls between these two lines, neither gentleman looses or wins any money.
I’ve highlighted 2016 with a vertical black line which is grey and a bit heavier in the “no win” region between the upper and lower grey dashed lines. Data are shown with open circles.
To assess who I think is going to win, picked a whole bunch of candidate models. In this particular instance, I used the “modified picking out of a hat” method. Based on general physics, I anticipate time will be a good proxy for the warming effect of GHG. So, I include time in the regressors. I have no idea what else to pick, so that’s it. I then arbitrarily selected first through 5th order polynomials and a Gompretz fit as candidate models.(Those who think there may be a slowly oscillating component in the data will recognize that their favorite candidate model is missing.)
For purposes of discussion, I’ll show how each model fits the data starting with the linear regression:
The linear fit is shown in red. The solid line indicates the “best fit” value. The upper and lower dashed curves indicate the upper and the lower 95% confidence intervals assuming the linear model is correct, but including the uncertainty in our fitting parameters.
Note the mean value for the linear fit (solid red slanted) lines just above the upper horizontal dashed grey line 2016. If we assume residuals to the linear fit are gaussian (which they may not be) this would mean that if the linear fit was “true”, Rob would have roughly 1/2 a chance of having to fork money over the William; the estimated value of 52% is shown in the red message in the lower left of the figure above. Meanwhile the chance of the ice falling in the “no one wins” regions is estimated at approximately 48% and the chance WC would need to fork money over to RD is 0.2%.
But suppose we evaluate probabilities using a quadratic fit.
If we believe the quadratic fit, the most likely outcome in 2016 is for the September ice extent to fall in the “no win” zone. If I assume the residuals to the fit are Gaussian, I estimate the chance Rob will have to pay WC $10,000 is 4%. Meanwhile, the chance WC will have to pay Rob is 23.5%.
It should be readily apparent now that the choice of model has a huge impact on the estimate of the odds. If someone thinks the linear line is “true”, they will be expecting a 50% chance Rob loses $10,000, a nearly 50% chance no one forks any money over to anyone and a very small chance WC forks over $10,000 to Rob. The chance Rob loses $10,000 is 4% and he’s got a 23% chance of winning the money. It should be readily apparent now that the choice of model has a huge impact on the estimate of the odds.
But which model is better?
Using the eyeball method, the quadratic fit appears “better” than the linear fit. The “goodness” is confirmed by noting the quadratic term in the quadratic fit is statistically significant and comparing the corrected Akaike values (AICs), which indicate if the only two possible models are linear or quadratic, the quadratic model much more likely than the linear model.
Now let’s look at cubic and 4th order polynomials:
The cubic is shown in mint green. If this is the correct model, WC has a 61% chance of having to fork money over to Rob while Rob has a 4% chance of having to fork over money to WC. Under the quartic fit, Rob has a 48% chance of winning $10,000 while WC has a 26% chance.
Supposedly. But it would be reasonable for someone to question the likelyhood that either of these fits is true because the some of the fit parameters for these regressions are not statistically significant. Ordinarily, I filter models with fit coefficients that are not statistically significant out of my weighted model because I think there is risk of introducing too much noise. But today, I’m going to keep them. With respect to evaluating the Dekker/Connelley bet, the main effect is to over state the probability that money is exchanged and understate the probability that no money will be exchanged. It may also make Rob a bit too confident he will win– but he’s already made the bet. If he wants me to recalculate throwing these out, I can do that. 🙂
My final fit was Gompretz which at least has the virtue that the best fit curve never falls below zero ice:
If this model is “true”, and I assume residuals are Gaussian, Rob has a 3.5% chance of forking money over to WC while WC has a 27.5% chance of forking money over to Rob.
Which model is best?
I wish I could suggest which model is best based on phenomenology, but the fact is I don’t know. That’s why I picked predictive models based on “modified picking out of a hat”. This method is dangerous. One danger is when used to extrapolate– as I am doing here– candidate models picked this way are likely to make Jeff Id’s head explode. However, if I were betting, I’d rather at least look at what extrapolation suggest rather than not looking at it. In that context, we will ignore the head exploding potential of the polynomial fits and decide which looks “best” based on the AICc criteria based on past data.
The AICc criteria and associated weights determined based on the AIC criteria are shown below:
| name | AICc | weights |
| linear | 53.93372 | 1.6% |
| quadratic | 47.70328 | 35.5% |
| cubic | 49.36520 | 15.4% |
| fourth | 49.36520 | 3.9% |
| Gompretz | 49.36520 | 43.6% |
According to the weights, if we assume this set of models contains the full set that “might” be “true”, the Gompretz fit is the most likely model and has a 43.6% chance of being “true”.
Applying my weights, I obtain a best fit model illustrated by the solid black curve below:
The dashed black curves show the ±95% confidence intervals assuming probability distribution of errors around the mean curve are gaussian. Under this assumption, the probability Rob will ow WC $10,000 is 8.5% Bear in mind: This lower interval was computed assuming that probability distribution function for the weighted model is normally distributed. This makes some sort of sense if we think of the candidate models as being samples drawn from a population of all possible “sane” models.
We could also interpret the weighted model literally: that is, we could assume there really are only 5 possible candidate models. In this case, even if the pdf of each candidate model is normally, the pdf for the weighted model is not normal. However, under the assumption the residuals from each candidate model are nomally, I can find the probability Rob must pay WC $10,000 by weighting the probability he looses under each candidate by the weight of that model. Doing it this way, Rob has a 5.4% chance of losing $10,000, meanwhile he has a 31.5% chance of winning.
Either way, it seems Rob has a very good chance of not losing any money. WC has enough of a chance of losing he should be a bit worried. The most likely outcome is no one wins.
Should anyone be contemplating side bets, they should bear in mind: though I presented this, I have not identified all possible plausible “candidate models”. Including outer models might change the outcome, but I can’t include them unless someone suggest a model form that might make sense. Using models with phenomenological support would be preferable to just looking at a range of polynomial fits. It may also be that considering modeling residuals with ARMA would make a difference. I haven’t looked, so I don’t know.
Another point to consider: we are approaching the 2011 extent minimum, and that value is likely to fall below level that would have been predicted by the best fit curve shown here. If this calculation is repeated in October, Rob’s chance of winning will look better than it does today.
Owing to my blunder, many reasonable comments are already on the other thread. But I think this post shows more detail, and may clarify my general procedure to people.
Now, it’s Saturday night. We’re having pizza. Hope your doing something fun too!
Given that RD has say a 20% chance of winning 10K from WC, perhaps somebody wants to buy his bet.
Rob’s chances are probably much better than any of these models predicts. That’s because ice loss can be severely non-linear: once it starts to go, it goes rapidly. The spring loss of ice cover on a lake is a good example: you get a little bit of open water around the edges for a few days, and then BAM! one warm (or windy) day, and you lose 90% of the ice cover all at once. (I live on a lake, and I see this every year.)
So rather than using any 2-D metric like ice extent or area, a much better predictor is ice volume. And if you look at the PIOMAS data for ice volume, we could see completely ice-free summers by 2016. No 2-D metric would be expected to model this, even using a cubic fit. But I think it’s completely realistic.
‘Under the most rigorously controlled conditions of pressure, temperature, volume, humidity, and other variables the organism will do as it damn well pleases.’
(Murphy Corollary.)
IIRC, Connelley says the modelling he has worked on shows that once the ice melts in summer, it just comes back again in winter, so he seems to believe there will be a limit to how the low ice can go by 2016. That is, the quadratic trend is not going to give the right answer due to the way the physical system works.
bugs–
Possibly, Connolley will turn out to be correct. I”m sure he already knew the trends didn’t look promising before he bet. To place the bet, he has to have a lot of confidence in some physical process and believe it will turn things around before 2016. Because the trends favor Dekker.
Of the fits I tried, the Gompretz fits best. That suggests the ice loss will slow down– but it still favors Rob winning.
All of these are just extrapolations based on fits to data– which makes them dangerous. If someone has a physically based type of fit, I’d add it. But I don’t know one.
Rob wanted to know how I might estimate– this is the general method I would use. I’m not claiming I have a huge amount of confidence in it.
If the physical system that influences Arctic ice includes the AMO, then WC may have just cause to be worried.
Hi Lucia,
Pedant’s point. I think that should be “Gompertz”.
For what it’s worth, I think that trying to fit to the timeseries without understanding the covariables, while entertaining, is not likely to make any punter rich. But, of course, you already know that. (smiley face)
Paul
Paul_K–
I think fitting the time series can help some people be more cautious about bets– and that includes those who think they have physical understanding, but actually don’t.
Sorry I’m too lazy to go back thru’ the history of this. Can somebody quickly explain who these two guys are? And why you think this bet will actually be seen thru’ to the end?
Lucia, thank you very much for taking the time to analyze (and correct) the odds of this bet considering various curve fittings. Your analysis confirms my own findings that I seem to have better chance of winning than William does (hee, otherwise I would not have engaged), even though we used different reasoning to obtain that conclusion.
I must have looked at the expectations for 2016 extent from a dozen angles, most of them physical effects.
Each time I came to the same conclusion :
Arctic will amplify a long term trend, because of positive feedback mechanisms, most of which are known effects with a physical explanation. Evidence of such positive feedback mechanisms are very clearly observable, such as the Arctic amplification effect :
http://www.skepticalscience.com/What-causes-Arctic-amplification.html
That’s just one of them (and there are at least 7 or 8 such known positive feedbacks working in the Arctic), but for trend development (my main interest) I realized that ANY positive feedback means that a long term linear trend should accellerate (exponentially). Since the first polynomial factor beyong linear is quadratic, we should initially (and probably until 2016) see at least a quadratic fit.
Longer term, negative feedbacks (like increased IR radiation due to higher SST, and heat not getting to remaining small ice pack) will take over, resulting in a ‘soft’ landing (similar to a Gomperz curve).
Another reason that the Gomperz curve may be the best fitting curve is that the best models of the Arctic show these positive and negative feedback mechanisms, and result in Gomperz curve decline as well. Look at the black line of the IPCC projection :
http://www.realclimate.org/images/seaice10.jpg
So from a physics point of view, the Gomperz curve (initially simply quadratic decline) makes a lot of sense.
What remained was this : If the models understand (actually calculate) these feedbacks so nicely, then why do the IPCC model projecting decline to set in so much slower than reality and observation suggests ?
It took me a while to understand that, until I read Serreze et al 2006 :
acacia.ucar.edu/cas/Trenberth/trenberth.papers/2006JD008230.pdf
Here is became blatantly clear what’s going on : The Arctic is very sensitive to minor energy input changes, at the level of single W/m^2. Insert a bit more power (for example from under-Arctic deep ocean convection) and ice declines faster. Insert less energy, and ice grows. On top of that, our observations are limited to an uncertainty in the range of 10’s of W/m^2.
So the models are fine, and the trend they project is fine, but we simply do not how fast that projected trend will develop since we cannot constraint the parameters and the strength of some of these positive feedback mechanisms accurately enough to make a clear projection.
That’s when I realized that if the models underestimate the feedback strengths (such as albedo effect and/or effects of salt-rejection deep ocean heat convection and/or effects of being the main ones) that Arctic sea ice decline trend projected by IPCC models may be realizing itself much faster and much earlier than anticipated.
Regarding the actual odds : most studies (and model trends) suggest that during the steep decline part of the Gompertz curve, that variability increases quite substantially. So I expect that William’s odds of winning are in reality much higher than the 3.5 % that come with assuming a ‘normal’ distrubution on the Gompertz curve. Incidentally, higher variability also increases by odds of winning under the same curve.
Final note : Regarding this variability, I expect negative feedbacks (increased IR radiation in later summer/fall) to kick in if a particular year falls below the trend line (and further decline if it’s above the trend). However, since 2011 4.4 or 4.5 minimum is pretty close to the quadratic trend that I expect, 2012 ice extent can still go either way, depending on weather. I DO however think that we will get another pretty warm and wet Arctic fall, and consequently another snowy winter in much of the NH, simply because the Arctic has to get rid of that heat it accumulated.
OK. That’s my two cts.
Sorry for the typos. It’s been a long day. I hope my main message comes through clearly though : Identifiable positive feedback mechanisms and an Arctic sensitive to energy forcing are consistent with the quadratic decline in sea ice extent that we observe. A quadratic decline which will soften out only when it is significantly smaller than today, and below the 3 million km^2 I anticipate in 2016. Variability will be high during that decline.
Rob, thanks for the explanation of your reasoning, especially your discussion of the relative merits of linear, quadratic, and Gompertz projections as applied to Arctic ice.
Here are the April sea ice numbers from the NSIDC (Area and Extent, NH, SH and Global)
What trend is evident here?
http://img859.imageshack.us/img859/2115/seaiceglobalnhshapr11.png
What about September? Well, there’s more of a change than April but I don’t see any quadratic trends in any of the actual numbers (the SH is even increasing).
http://img24.imageshack.us/img24/5830/seaiceglobalnhshsept10.png
Bill– Here, I zoomed in to focus on the data we’ve been discussing. Removing the irrelevant data that unnecessarily expands the Y axis and squishes the NH extent data; I’ve unsquished to make it easier to see the variations:
Rob
Considering physical effects is — at least in principle– better than curve fitting. I think curve fitting is a good supplement. After all, if one’s notion of physical effects cannot be made to square with the data, then it’s plausible that something is missing from the physical explanation.
I’d have to do something much more complicated to include the possibility of systematic changes in the variability. So…. I didn’t. As I think many know I wasn’t planning to post this. But I made a comment — probably having mistaken the date of your bet– and then you asked, so I did post a way of looking at it. I can’t begin to pretend this is the be-all and end all.
I’ve also thought about the fact that I don’t include the possibility of autocorrelation in the residuals in the current method. I suspect (but am not sure) that if I included those with the linear fit it might find a linear fit that suggested the possiblity of recovery with a higher weight assigned by AICc, and then the method might estimate Connolley having better odds. But I haven’t done that. So I can’t tell you what I would find if I considered that. (I’m not sure it’s worth doing since (a) you two already bet and (b) I don’t think anyone is really using these to place bets. But it’s worth knowing when deciding how ‘sure’ you are should you wish to bet in the future.)
Yes. When the best estimate is in the “no one wins” region, high variability increases the likelihood that money will be exchanged. I suppose there is someway of looking at the data to see if the notion that variability has increased in panning out, but I haven’t given it much thought. It would be worth doing if someone was really using this sort of thing to gauge their odds.
As the queen of typos, I can hardly complain.
lucia (Comment #81105)
September 3rd, 2011 at 7:32 am
—-
Hmm, well that’s interesting.
Bill–
There is a possibility the loss rate isn’t quadratic– or Gomprets or anything else. But if you are trying to estimate odds for a fairly short term bet (i.e. 2016), I think it’s worth knowing what curve fitting suggests. It’s not physics– but then betting isn’t really physics, is it?
Lucia,
Considering physical effects is — at least in principle– better than curve fitting. I think curve fitting is a good supplement. After all, if one’s notion of physical effects cannot be made to square with the data, then it’s plausible that something is missing from the physical explanation.
I agree. In fact, I think that curve fitting, especially when done coutiously with a weighted model as you did, is putting constraints on physical effects. For example, Serreze et al 2006 metions that the Arctic energy budget is constrained only to 10 W/m^2. This means that we can’t really determine ice growth/thinning more accurately than some 0.5 meter per year or so. Analysis of past extent/volume therefore is very valuable it putting a constraint on the rate of extent or ice volume decline. Somewhat equal to “back to reality”.