It all started with Cliff Mass saying “… with huge transient peaks and troughs (see below). With such variability …” when talking about CalFire data of area burned by wildfire in California from 1987 through 2016. His comment led me to make a very serious mistake: I thought he might actually know what he was talking about.
I figured he must be talking about the fact that the “huge transient peaks and troughs” indicated that the noise in the data was severely non-Gaussian. That would weaken a trend test like ordinary least squares (OLS), which might suggest a significant trend when there is none or fail to detect a significant trend when the data actually demonstrate it.
So I did OLS, then subjected the residuals to the Shapiro-Wilk test for normality. Result: they definitely don’t follow the normal distribution, and when you can show that with just 30 data points you shouldn’t ignore it. Fortunately, there are many robust trend tests. My favorite is the Theil-Sen method, so I used that. Result: the trend is statistically significant.
There are other ways. For example Keeley & Syphard, when analyzing annual data, decided to log-transform the data and anlyze those values. Just for fun, I did the same with the CalFire data. Result: the trend is statistically significant. The p-value is 0.0462, that’s 95.4% confidence. Also, the Shapiro-Wilk test applied those residuals does not demonstrate departure from the normal distribution — so ordinary least squares is not a weak test.
But if you use ordinary least squares on the raw data the p-value is 0.0722. That’s significant at 92.8% confidence, but not at 95% confidence.
Here’s something for Cliff Mass to think about: why is it that when you use a test which is demonstrably weak it doesn’t give 95% confidence, but when you use a test which is not demonstrably weak is does give 95% confidence?
Cliff Mass finally got around to doing a statistical test, after he had drawn a conclusion with no test at all. Here’s what he has to say about it:
Cliff Mass said…
Several of you are asking about the significance of the trend in acreage (e.g., Sofistek). I did the trend significance analysis today, using the same approach I and other used in our peer-reviewed paper on snowpack trend (found here, https://journals.ametsoc.org/doi/abs/10.1175/2009JCLI2911.1, based on the approach of Casola et al., 2009). This type of trend analysis takes in consideration the variability of the time series (which is very important). I found that that trend over the entire period was NOT statistically significant (generally we used the 95% level…a trend is considered significant if there is less than a 5% chance of it happening by chance).
A global warming activists (someone named Tamino) claims otherwise, but his method is not appropriate (I won’t get into the technical issues here). He also likes to mock folks he disagrees with, which is not the way scientists interact with each other….cliff
August 12, 2018 at 5:17 PM
If you read the paper he refers to, then the one by Casola et al., you discover that the approach used is: ordinary least squares. The only thing Casola et al. do to it is combine some algebaric manipulation with a simplifying approximation (which weakens the test but not by much).
So the method Cliff Mass used was: ordinarly least squares. Seriously. That’s it.
Now for the truly bizarre part: Cliff Mass keeps repeating that “This type of trend analysis takes in consideration the variability of the time series (which is very important).” Cliff, they all do. Least squares, Theil-Sen, L1 regression, Mann-Kendall, you name it — they all do.
When Cliff Mass talked about “the variability of the time series” I thought he was talking about something other than the fact that there’s noise in time series and every statistical test takes that into account. Giving Cliff Mass too much credit — that was my mistake.
Here’s my prediction: Cliff Mass will probably not respond to this criticism of his faulty analysis, but if he does, he’ll be sure to end his comment with an ad hominem comment about me personally.
This blog is made possible by readers like you; join others by donating at My Wee Dragon.