Lately I’ve been having fun with change-point analysis.
There are many versions, depending on what kinds of change you allow. The most common is to allow a change in the value of the data at some particular time. Essentially, one creates a step-function model in which the value is constant within each time span, but from one span to another the step occurs. How many times this happens (how many spans there are), and what the values are during each span, are estimated by change-point analysis. Perhaps most important, it applies some rigorous statistics to determine whether or not the changes are statistically justified.
That’s quite necessary, and definitely not trivial because you can’t just go looking for the change-point time which gives the best fit, then evaluate that fit in isolation. When you have many possible change-point times and you test them all (or at least, almost all), you get so many chances to find one that fits well just by accident that you have to compensate for all those chances you took. After all, hitting the bulls-eye isn’t nearly so impressive if you’re allowed to take a million shots to do so.
Another form of change-point analysis looks not for a sudden change in the value of a time series, but in the slope of a best-fit model. In this case we assume the underlying model is a straight line over each time segment, but at the change-points the slope is allowed to change. Even so, the segments have to match up at those change-point times, so our model is a continuous piecewise-linear function of time. Continuous is what ensures that it doesn’t suddenly “jump” from one value to another (although the slope is allowed to jump from one value to another).
Then there’s the version in which both the value and the slope are allowed to change at the change-point. This still gives a model which is piecewise-linear, but it’s no longer necessarily continuous. Sometimes such models are counterindicated by other considerations; for instance, on physical grounds we really don’t expect something like temperature to suddently jump from one value to another, we expect it to change by the flow of energy into the system, which implies a continuous change and therefore a continuous model.
An important thing to bear in mind is that, although change-point analysis (in all its forms) creates a good model of a time series, one which is statistically justified in that the changes are reliably real rather than random accidents, that doesn’t mean that the model is right. Remember the words of George Box: “All models are wrong, some models are useful.” After all, we really don’t expect the trend in something like temperature to follow a perfectly straight line, not even over a number of different pieces.
But if change-point analysis gives us a decent model like that, we can be sure that the piece-wise linear model is a good approximation. We can also be confident that where changes occur, they’re real. As for other changes which are different from the model, they’re almost certainly there but we can’t be confident that they exist and we can’t justify saying when or how they take place. There are likely other changes indeed, but they’re bound to be small compared to those we can confirm, and we don’t have enough information in the data itself to justify such claims. The piece-wise linear model from change-point analysis probably isn’t all that’s going on, but it’s close to all we can deduce from the given data.
Let’s begin with the version which looks for change in the trend rate but not the value.
We’re rapidly approaching the summer minimum of Arctic sea ice (which generally happens in September). Some have claimed that since its precipitous decline, reaching an unprecedented low in 2012, Arctic sea ice has been in “recovery.” It ain’t so; those who say it’s recovering are either willfuly deceptive, or woefully ignorant.
The National Snow and Ice Data Center has released their monthly data through August, so let’s take a look. Here’s the latest graph of anomaly values (to remove the rather large annual cycle):
I’ve added two different smooths (thick lines in red and blue), more clearly to emphasize what the trend might be. The difference in the smooths is the time scale on which they smooth. The red line shows steady decline, but ice loss accelerated around 1998. In fact both smooths agree well up to that point. But after 1998, the “slower” smooth (red) just goes steadily down while the “faster” smooth (blue) first declines rapidly, then less rapidly, seeming even to level off. Which is more correct? What can be justified statistically?
Change-point analysis to the rescue! Using the trend-rate change-point version, we find not just one but two change-points that are statistically justified, giving a model like this:
Arctic sea ice loss did indeed accelerate, around 2002. Then it went through a period of extremely rapid decline, but about 2006 it returned to a slower rate of decline, in fact it may have stopped declining, becoming somewhat “stable.” If we look at the rates determined by change-point analysis, we get this for the three episodes:
The estimated rate of change since 2007 is negative as before, but is not statistically significant, so it is indeed possible that Arctic sea ice has stabilized (but a million and a half square kilometers below its prior value) since then. But it’s also possible that it has continued to decline since then, at the same rate it was declining prior to its period of hyper-active ice loss.
Even if it has stabilized since 2007, sea ice extent is still below what it would have been had it continued at the same rate since the beginning. So, in no sense can this be called some kind of “improvement,” and the notion of “recovery” is downright foolish. The Arctic sea ice is still in deep trouble, as the Arctic continues to warm. What will happen in the near future, only the near future can tell for sure.
For quite a different example, here are poll numbers among Republican voters over the last six months, for presidential candidate Donald Trump:
One can smooth the results on many different time scales, but one which seemed to me to be reasonable looks like this:
It makes it seem that Trump’s support among Republican voters took off at a rapid pace, then the pace declined (although, only a little). The same change-point analysis we used on Arctic sea ice gives this:
Which justifies the chosen time scale for the smoothing, and supports the idea of very rapid increase followed by somewhat less rapid increase. But is that the whole story?
Popularity among voters can change suddenly, so it need not be a continuous function of time. The moment when Trump’s popularity “took off” coincides with his announcement for his presidential campaign in June, when he made the now-famous comment about Mexicans (which rather displeased both Mexicans and Mexican-Americans). Could his popularity have changed both its trend rate and its value at that moment?
The version of change-point analysis which allows both value and slope to change, confirms that indeed it did:
It appears that Trump’s June announcement, the one that caused so much ire among so many people, also caused a sudden increase in his popularity among republican voters. His popularity has further increased, apparently steadily, since then, with him now holding a big lead over the overcrowded field of republican candidates.
There’s great value in the many forms of change-point analysis. As for Donald Trump, you are invited to form your own opinion.
Open Mind 
Concerning the Arctic ice story, it would probably help if you analysed each season independently, rather than trying to fit the entire series. You’ve already proved they’re behaving differently, with the annual cycle getting wider, so why roll them back together?
I’d like to see the separate fits for Sept/Dec/March/June – which will almost certainly show that March has stabilised, June and Sept continue to accelerate downwards, and Dec is more mixed.
That might also avoid giving ammunition to the denialist crowd. This current article will almost certainly be used to say “Prominent AGW statistician says that Arctic ice loss has stopped”, which is a shame and rather an own goal.
I suspect that it’s quite hard to do this sort of analysis with only a small number of time points. The uncertainties in any trend estimate will be very large with only five annual averages in the period 2002-2006 (the middle period with a different trend in the analysis above) and so it becomes hard to say with confidence that the trend has changed.
I agree with you that there are good physical reasons to suspect that you would have different trends for different seasons, but I still think the analysis is interesting.
Further, as tamino has shown with the global mean temperature data, we should not be surprised to see temporary apparent lulls in a trend, if one has a cyclical change (as with the ENSO cycle for global surface temperature) combined with a linear trend.
Finally, there has been some discussion over on Neven’s Arctic sea ice blog/forum about a new paradigm for Arctic sea ice where the multi-year sea ice is much less likely to survive the melt season (and is mostly now gone), and as a result the ice pack is much more responsive to the weather conditions during the melting season, as the inertia in the system is reduced.
It’s possible that the change points that tamino has identified with this analysis match up with that. I note that the period 2006-2012 that he identifies as the most rapid decline, coincides with the Great Arctic Cyclone of 2012 at one end, and nearly with the big melt of 2007 at the other. I think that’s pretty much when all the multi-year sea ice melted out.
One observation/question to be answered by/in your proposal is in the definition of a “season boundary”. Sure, there are astronomical ways of defining it, but it might do to allow some flexibility in the definition of what constitutes a “change of season” around a season boundary. So, for instance, if I were doing this with a state-space model and using a Kalman/Rauch/Tung/Striebel model (after Durbin, Koopman, Ooms, and Harvey, for instance), I’d pin down the season boundaries to the astronomical, but then add 4 terms to be added to that boundary day (defining, say, each of the ends of the seasons) such that the terms summed to zero.
Tamino: I’m a follower (with great interest) of your blog, and I have a statistical question (not directly relevant to this post, but I’m unsure how else to make contact) that relates to a climate denier’s statistical treatment of lower tropospheric warming. This person regresses lower tropospheric temperature anomaly not on time, but on *CO2 levels*, and then does some statistical tests to show there is no significance to the relationship. He/she then goes on to make grand pronouncements about the falsehood of mainstream climate science. http://greenheretic.com/has-warming-been-statistically-significant/#comment-7
I can see some problems with this approach straight off: it ignores the dynamics as well as the nonlinearity of the relationship between CO2 concentration and temperature, and it throws measurement uncertainty for CO2 (which has nothing to do with its relation to temperature) into the mix. But I feel there is something deeper that is wrong with the whole approach that as a non-statistician, I can’t put my finger on. Perhaps it has to do with the validity of the assumption that the residuals in this procedure ought to be independent?
I’m someone who has educated himself in climate science and it’s rare that I can’t defend it against this sort of question. I’d like to hear you weigh in on this!
Regression against CO2 isn’t too important here since the CO2 trend is much larger than the seasonal cycle. However, he first differenced the data. Of course there is no significant trend leftover, because first differencing makes series stationary. This is really all that needs to be said, but I’ll go on.
He didn’t check the information criterion either; a model that allows a trend component (in R this would be equivalent to specifying an “xreg” vector when making the arima() model) has a lower AIC and BIC for UAH data so it is a better model than the first difference model. Even an AR(1) model has a lower AIC/BIC than the first difference AR process.
When checking to see what models may work it helps to plot the series you’re looking at, and its ACF/PACF. For starters, if you see a trend in the data, then you *investigate models that have trends*, especially contrasted to stationary models. The trend is very obvious in the UAH temperature plot.
And the PACF of the UAH clearly shows we are most likely working with an AR(2) process since the AR(1) and AR(2) figures are significant; there is not very good indication of an ARMA process but we can compare them anyway. AR(2) with trend ends up being the best in terms of the most white residuals, lowest AIC/BIC, largest Ljung-Box p-value, and FWIW a very significant positive trend. If Tamino or anyone else would like to verify please do, but I think my summary is correct here.
To elaborate on a point: you can specify “xreg” to be the CO2 series over the same period, but if you include first differencing the trend component that shows up is the trend in the differenced series, i.e. will usually be insignificant as he found. If you let “xreg” be NULL the AR(1) value does not appreciably change, which really just shows how unimportant the trend in the first differenced series is.
AlexC <>
Question, AlexC: Did you PASS intermediate statistics? Had you been in the class that I TA’d, I would have failed you on that piece of your final exam. You don’t even understand the rudiments of an ARIMA model.
When you difference, you don’t eliminate the trend in any way. The trend ‘moves’ from a coefficient to become the constant, aka ‘drift’ in time series vernacular.
GreenHeretic:
“When you difference, you don’t eliminate the trend in any way. ”
By “eliminate” I of course mean “does not maintain the trend as a trend itself”, but this discussion has highlighted where I went wrong in interpreting what you were saying anyway. I had misunderstood you to mean that that figure was the trend component of the first difference series itself; I see now on closer reread that you had indeed meant that it was the mean of the difference series itself, adjusted for autocorrelation. This was my bad.
Since R does not explicitly give that number, as the program you are using does, I could just mimic by calculating the mean of the residuals of the ARIMA(1,1,0) model itself. I get the same value of 0.0017, though the standard error I calculate is a bit less than one half the value that your program gave, at 0.0054 (I am unsure why the difference would exist).
So I understand now what you are getting at and why you have concluded what you did based on your choice of model. However, I think that your choice of model is incorrect, for the other reasons I have listed here, as well as what Tamino has said with regard to the Pierre-Perron test. Insofar as my comments have suggested that you calculated a trend *after* first differencing the series though, I will attach a small note as a reply to those comments correcting my mistake. I believe it is only one other comment right now, on the later blog post.
AlexC
The arima() function in R forces the mean (mislabeled as the intercept) to be zero. To correctly model a time series with a differencing term, you should use the command
arima(diff(data), order = c(1, 0, 0))
rather than
arima(data, order = c(1, 1, 0))
Then the function will return the mean and what it computes as the standard error.
The computation in R for the standard error agrees with yours. (I assume you computed this as the standard deviation of the residuals divided by the square root of the number of data points.)
However, this is not correct, because it does not take into account the correlations in the original data. Even though the residuals are uncorrelated, the fact that the original data was correlated means that the sample variance of the residuals divided by the sample size is no longer an unbiased estimator of the variance of the estimator of the mean.
In “Time Series: Theory and Methods”, chapter 7, by Brockwell and Davis, they give the correct estimator for the variance as
1/n ( sum [ (1 – |h|/n ) gamma(h) ] )
where the sum is taken over all values of h from -n to n and gamma(h) is the autocovariance at lag h of the differenced process.
We can use the estimated values of the covariances of the differenced process in this formula, which we can get from R with the command
acf( diff(data), type=”covariance” )
The fact that the estimated value of the AR coefficient is negative means that some of the autocovariances are negative, which in turn results in a smaller value for the standard error.
I don’t know how GreenHeretic’s standard error estimate was computed, but it is clearly wrong.
MrMath:
Thanks for the correction. Yes, that is how I calculated the standard error; thanks for the equation as well, I’ve had a chance to find an online version of the book and see the equation you mention. Using that equation and the code suggestion you gave, I get a standard error of 1.644E(-5), to be tacked on/off for the 0.0012 term.
I find myself still confused for the difference between the mean values that GreenHeretic got and what the ARIMA(1,0,0) for the diff(data) gives.
Sorry, I double checked my code and see I made a couple mistakes: didn’t sum from -h to h as you said, and took that variance to be the standard error. So fixing the first part and taking the square root, I obtain:
0.0012 ± 0.00112 (2 s.e.)
So the warming is statistically significant in this model. And I’m ready for some sleep now, played enough with R today (including simulating unit root models v. AR(1) models and how they do or don’t diverge over time, as per what Tamino said elsewhere—very interesting graphs IMO).
I would make two comments. First the claim that satellite data is high quality is dubious. Satellites do not measure temperatures directly, they use microwave radiation as a proxy for temperature. Furthermore they look through the stratosphere, which is cooling to see the lower troposphere. To handle this they use models with a number of inherent assumptions which they then implement in large quantities of Fortran code. In the case of the UAH series John Christie, one of the two main developers of the code does not use subroutines and has allegedly described the computer software he helped develop as “spaghetti Fortran”. Roy Spencer, the other hand does use the 1960’s innovation of subroutines. In any case people who have attempted to read the code describe it as impenetrable having large blocks of code which are commented out and others which are unreachable. Change controll if it exists at all appears to be very sloppy – and there have been a number of significant revisions. The UAH satellite estimates of temperature and the RSS satellite estimates (using the same raw data) are significantly different, which reflects the difficulty and uncertainty of this method of temperature estimation.
More importantly a failed statistical test proves nothing apart from the fact that the test is too weak to detect any relationship which does exist. This is a fundamental truth which researchers often fail to understand. Statistical tests can only disprove the null hypothesis (or more precisely show that it is unlikely to be true), they cannot prove it.
I should have made it clear that the above comment was directed to Patrick Shoemaker – not Tamino’s post
Your commentary on what satellites measure is misplaced. How is that any different in principle than any temperature measurement?
As for the ‘failed statistical test’ proving nothing, you really should take remedial stats. Nobody claimed that the null hypothesis was ‘proved’, only that it could not be rejected, and that the alternative hypothesis could not be accepted. Could the minor warming we have seen been the result of random variation? The numbers say, EASILY.
At the very least, the lack of statistical significance demonstrates that the debate over global warming is not over. If you want to end the debate, fine. There’s nothing there.
Patrick,
The first thing to notice is that GreenHeretic is engaging in misdirection. The starting point is the claim that the UAH satellite record of lower troposphere temperatures is “very high quality satellite data” and “there is no reason to suspect any substantive differences between the surface and the lower troposphere that lies directly above it.”
In fact the surface temperature records have at least as good a claim of being high quality as the satellite data. And _every_ surface temperature record shows a higher rate of warming over the period 1979-2015 than the satellite lower troposphere records. It is reasonable to ask why this discrepancy exists, but by using the UAH data and claiming that there’s no difference with the surface temperature, GreenHeretic is putting a thumb on the scale.
Next, the article points out that a linear fit to the data shows a statistically significant trend. (Using R, I get a trend of 0.0140 degrees Celsius per year, with a p-value indistinguishable from 0.) It then goes on to assert that we should be regressing against CO2 concentration rather than against time.
This is mostly a red herring. CO2 concentration has increased almost linearly since 1979, aside from a small seasonal variation, so the fit to CO2 concentration would be expected to be quite close to the fit to time, and in fact it is.
I suspect this bit of work is to justify the claim that increasing CO2 has no effect on temperature, rather than the initial claim that temperature has not been increasing in time. Even if the other claims are justified, this reasoning doesn’t really hold because we should expect that there are substantial time lags in the system. Even if equilibrium global temperature is linearly related to CO2 concentration, we may not see a linear relationship between global temperature and CO2 concentration while the concentration is changing and we are not at equilibrium.
The CO2 digression aside, GreenHeretic points out that linear regression is only justified if the noise is uncorrelated, and in this case the noise is clearly not uncorrelated. Rather than using a simple linear model, an ARIMA(1, 1, 0) time series model is more appropriate. No argument from me on this point.
The stated model is:
Intercept: 0.0017
AR Coefficient: – 0.3359
Standard Deviation: 0.1131
GreenHeretic also reports standard errors. The reported standard error for the AR coefficient is 0.0113. When I recompute the model in R, I get a standard error for the AR coefficient is 0.0450. I do not know why (or how) these numbers could be so different. The difference is in the other direction for the intercept, with standard error from GreenHeretic of 0.0135, compared to the standard error from R for the mean of 0.0040. Again, I don’t know how or why these are so different.
This doesn’t actually solve the problem. The standard error is still sufficiently large that a hypothesis test will not reject the conclusion that the intercept is zero. (On the other side, it also can’t reject the hypothesis that the temperature is increasing at a rate of, say 0.0075 degrees Celsius per month, or 9 degrees per century.) On the other hand, the time series literature that I’m familiar with never refers to hypothesis tests of the intercept. I’d be interested in any references, if anyone out there has one. I’m not convinced that it is an appropriate test.
Assume it is an appropriate test, and also that the intercept really is 0.0017 and the standard deviation of the noise really is 0.1131. We can estimate the number of data points we would need in order to conclude that the mean is statistically different than zero. We get that we need 17000 data points. At one data point per month, we would have to wait 1400 years, by which time the temperature would have risen 21.6 degrees Celsius.
My conclusion is that this test is unable to reject the null hypothesis even when the null hypothesis is clearly false.
Meanwhile, GreenHeretic’s fundamental claim is that there has been no global warming since 1979. The last couple of paragraphs try to hedge this claim, effectively turning it into (paraphrasing) “global warming models say that there should have been statistically significant warming since 1979. Since there has been no statistically significant warming, the models must be wrong.”
The first objection is that GreenHeretic hasn’t actually demonstrated that the models should predict statistically significant warming, according to this test. The second objection is that there is lots and lots of evidence that increasing the concentration of CO2 increases the global temperature, humans are increasing the concentration of CO2, and global temperatures are increasing. If this one test fails to confirm this increase in temperatures, it does not mean that there is no other evidence and we should conclude global warming isn’t happening.
<>
You seem a trifle confused.
<>
I never said that. I said nothing more than that there hasn’t been any statistically significant warming since 1979. Please keep in mind that most Deniers say that there hasn’t been any warming since 1998. I am including the entire satellite record, not cherry picking by time period.
<>
Fair enough.However, at the very LEAST, the pronouncements that the debate is over are premature. In fact, the test shows that the slight increase we have seen is not even close to statistically significant and could easily be the result of chance.
<>
Then, SHOW ME with real numbers that can be checked that global temperatures are increasing at a rate that cannot easily be explained by chance. Should be easy-peasy if you are right.
Actually, the Deniers don’t claim that increased CO2 levels won’t increase temps. They agree that the increase is greater than zero. That is where the 97% consensus canard comes from. Did you know that many leading deniers are on that list? What they do NOT agree with is the notion that there is any sort of amplification mechanism operating. Deniers say that the temp increase will be trivial. The actual data confirms the Deniers.
[Response: First of all, the study of actual data by those who know what they’re doing does not “confirm the Deniers,” it contradicts them. Time and time again. Your claim is so ludicrous it’s almost funny.
Second, this blog is for rational people. It’s time for you to return to your haven for those in denial.]
A shorter response to GreenHeretic’s test is that it proves too much. Taking the numbers at face value, the conclusion is that the temperature could have increased by 6 degrees Celsius since 1979 and the test would still conclude that there was no statistically significant warming.
I think GreenHeretic’s standard error is not correct. But using the value I computed, there still could have been 2 degrees of warming since 1979 and it would not be statistically significant.
Statistical significance is not always the appropriate test to apply to the data.
“Statistical significance is not always the appropriate test to apply to the data.”
Pray tell… what tests would YOU apply?
The problem is that your test has no power. If your standard error is correct, the data could show 6 degrees of warming since 1979, and your test would still report that the warming was not statistically significant.
At that point, your test can’t prove anything, so there’s no point in using it.
By the way, what software did you use to perform this analysis? Does it state how the standard errors are computed? Does it justify using those standard errors for hypothesis testing?
The reason I ask is that as I said above, when I estimated the ARIMA(1,1,0) model in R, I got basically the same coefficients that you had, but a substantially smaller standard error.
I think I know how R is computing the standard error, but I referred to Chapter 7 of “Time Series: Theory and Methods” by Brockwell and Davis, which derives the correct formula for the standard error. R does not use their formula. When I use that formula, the standard error is smaller still.
In fact, using that calculation of the standard error, the warming is statistically significant.
I’m really not smart enough to be empirical about this, but it smells fishy, like in a red herring sort of way. With musty overtones of strawman. Showing off your regression test as you whale on a straw herring provides a distracting sleight of hand, not much else.
How hot would the lower troposphere need to be to pass the test? Who predicted the lower troposphere would be that hot for a given amount of CO2? If you’ve not got sensible answers to those questions then… …what are we testing for again? Why are we caring? As Paul P says, what exactly is the null hypothesis here…
Dubious claims that satellite data is ‘very high quality’ (my understanding is it has been and continues to be non trivial to infer temps at a specific altitude from far away in space) and that there’s no reason to think the Lower Trop would much differ from surface temps cos it’s nearby (really?) and, oh, the rest of the blog don’t inspire great confidence I’m afraid. But mostly I’m not sure that the assertion – that anyone should worry about this – isn’t just plain fallacious to start with.
But if someone with actual knowledge was inclined, I’m sure they could have brief fun with it.
Patrick Shoemaker: Tamino will probably have some choice words about “GreenHeretic”‘s analysis you referenced.
But in the meantime you might like to see the analysis done correctly, which Tamino discussed back in late 2011/early 2012. See the article in Environmental Research Letters by G. Foster and S. Rahmstorf (“Global temperature evolution 1979–2010”, open access via the DOI proxy server at http://dx.doi.org/10.1088/1748-9326/6/4/044022).
“GreenHeretic” is a poor analyst because he/she did not even look for other influences that would impact the TLT (UAH) data product”s residuals from a linear trend fit: Specifically, the very prominent stratospheric-injecting volcano eruptions in early 1980s (El Chichon) and early 1990s (Pinatubo), the small variations in Total Solar Irradiance (TSI) due to the solar activity cycles (at about 11-year periods), and the known temperature modulations due to the ENSO phenomena. All the statistical buzz phrases thrown about by “GreenHeretic” cannot make up for the lack of Heads-Up analyis that work displays.
This is also a kind of change point analysis, inspired by Tamino’s blog post “Desperate for a pause”:
https://drive.google.com/file/d/0B_dL1shkWewaNDVmS0t1bjZjQXM/view?usp=docslist_api
Yes, I know, there are simplifications, no RATPAC mask, no transformation to TLT weighting functions, SE straight from LINEST function, etc. The change point is based on meta data, it is convenient with 2000, but I think that a change point analysis based on data would give 1999.
However, the findings seem to be robust. UAHv6 can be replaced by RSS or UAH v5.6, land only or sea only, and RATPAC can be replaced by NCEP/NCAR or ERA-interim 850-300 hPa, with essentially the same result.
Something is seemingly wrong with the satellite series after the turn of the century, thus explaining “no warming for xx years”. The temperatures run about 0.3 C too low today in the satellite data sets, making it less likely that the coming el Nino will give new temperature records by year or by month. However, there are surely new global heat records coming up in surface, reanalysis, and radiosonde datasets…
My ISP lost the longer comment so just saying thanks for this. Somewhat uplifting to see the revised models of sea ice loss might have it better this time. Moved my guess over this to the direction of mainstream estimate.
Ed Hawkins post on the current state of sea ice models, it looks like there’s been significant developments since AR4, and even after AR5 deadline: http://www.climate-lab-book.ac.uk/2015/the-sea-ice-orchestra/
Thanks to the commenters regarding the critiques of the “GreenHeretic” post. And … I found the change point analysis of Trump’s popularity amusing. I’ve been wondering if a big, blowhard statement about the commie conspiracy of climate change might be forthcoming, and if that could kick off another change point for him — but I reckon (hope) the supply of angry dimwits he’s aiming to capture may be nearing saturation.
I would suspect that lots of zeros for trump prior to announcing his candidacy would be because he wasn’t offered as an option. Averaging such zeros and positive numbers in this period would seem to be methodological differences and not meaningful. Shouldn’t these data points be ignored as (if?) they are unable to provide information?
The trend line before announcements would then be noticeably higher reducing the size and, I would think, significance of the step.
[Response: Right you are. I figured this out recently while studying the data for Joe Biden, who still hasn’t declared candidacy, and comes up clear 3rd for the democrats in polls where he’s included but of course dead last in polls where he’s not offered as an option. There’s also the fact that the numbers for his rivals (mainly Clinton and Sanders) are also affected by his presence as an option. Back to the drawing board for me.]
Nice result on the sea ice.
Peter Ellis suggests looking at different seasons. My reaction would be to also test area and volume data. If the dates agree this would assist conclusion of significance. However, I guess it makes calculating significance tricky or perhaps even impossible. So when should such testing be extended to other metrics?
So with extent minimum this year of around 4.4 m Km^2, average for 2012 to 2015 about 4.6 m Km^2 and a downward trend ranging from -.04 to plus 0.005 m Km^2 per year we have a date range for trend reaching 1m km^2 extent of never to 92 years with a central estimate of about 183 years.
The above is of course obvious nonsense. If interested in September minimum we should look at September minimum data as September could be declining rapidly whilst March barely changes. If the break is after September 2006, using NSIDC September average data the trend is upward so the central estimate for date of trend getting down to 1m Km^2 is never.
Neither of above seem ideal. Two changes, one getting steeper then one getting shallower is obviously not enough data to suggest some oscillation and the next change of trend will be to steeper again even if this seems a likely possibility.
Ho hum – if it is good information against crazy alarmists predicting imminent collapse of sea ice, then it probably isn’t also going to be good against crazy denialists. Against denialists, I guess it is a case of saying short data trend since last break in trend rate is not a long enough period. Also rather than just using data we should also use model results and even without model results, we know GHGs provide insulating mechanism and upward trend in ocean temperatures. Therefore we know the trend in Arctic sea ice will be downwards over longer periods. If the data over too short a period since the last break says the trend is upwards (or very slowly downward), then our knowledge means the trend is likely still to be downwards and an apparent upward trend is highly likely to be either spurious or a cyclical upturn.
A better version of this is likely to be possible. At least some version should be present?
Thanks for introducing me to change-point analysis. I spent many years looking at time series data for industrial processes, trying to detect change points and correlate them to possible causes. I got pretty good with cusums, but never knew about the bootstrapping technique. Another new tool is always appreciated.
I’ve been thinking that since 2007 there’s been more snow due ,in part, to the loss in extent, and with more snow cover we end up with less mass even given similar volumes and extent. So whilst this metric says 2006 was the change point, the actual loss of ice [mass] continues and falls a little more quickly than it seems from available measures.