When I chose the title for the last post, I didn’t really intend to stimulate discussion of the Phil Jones interview. I just thought it was a catchy title for a post about the fact that if you account for exogenous factors, you can establish a trend with less data than you’d need without accounting for exogenous factors.
Nonetheless, a lot of commentary mentioned the Phil Jones BBC interview. And that caused me to ponder such questions as “What should Jones have said?” and “What would I have said?” In fact, since I hadn’t done my recent analysis at that time, I might have responded very similarly to the way Jones did.
It was mentioned that his response was scientifically correct, and scientifically appropriate. I agree. He gave the right answer, if his audience consisted of scientists. But of course, it didn’t. Therefore it’s fair to consider not only the scientific content of his response, but also its impact on public perception of global warming.
So I’ll pose the question to readers: supposing that warming since 1995 wasn’t statistically significant, what would be the ideal response to the question?
Open Mind 
To the loaded question “Do you agree that from 1995 to the present there has been no statistically-significant global warming?”, I would try to turn it into a teachable moment, with something like “When scientists say ‘statistically significant’, they mean there’s a 95% chance of something happening. In the case of global temperature, there’s a 93% chance that global warming has been happening since 1995. So will the trend doesn’t cross that somewhat arbitrary 95% threshhold, a 93% chance of warming is still pretty strong evidence”
Of course, put on the spot in an interview with a question out of left field like that, who knows what I’d manage to stutter out! And if I hadn’t put the interviewer to sleep by that point, I’d also like to point out that when you consider global temperature series like GISS, the chance of warming is even greater than 95% – partial records like HadCRUT missing out the strongly warming Arctic region and underestimate the warming trend.
Ah, but John, he must have had time to prepare an answer. How else would he have been able to answer “not statistically significant” so quickly?
Because he probably daily updates from Tony Watts and Dick Lindzen pointing this out.
“It is not appropriate to measure for statistical significance over a 15 year time frame – there has definitely been statistically significant warming since 1994.”
Looking at the HAD data on global temperature, the best estimate of the rate of global warming is 0.1 deg/decade . To an 80% significance level the trend is above zero.
In any context other than the poisoned well of climate discourse, his response was fine. Not just for scientists, but for any reasonably educated reader.
“but only just”
trend: “+0.12C per decade”
“not significant at the 95% level” – OK, here, he could’ve said at the 92.5% or 93% level (I forget exactly where the simple regression using HadCRUT lies, but it was about there). That would’ve greatly mitigated the damage, because non-tech people would say “OMIGOD! 92%? 93%? but not quite 95%? Still damned persuasive!”
That was the big mistake, IMO. Not to state the actual level so that lay people would understand the hair-splitting differentiation inherent in the question.
But, really, in any environment other than one where “climate science is a fraud!” is a dominant theme, his answer was very reasonable, not just for scientists, but for intelligent laypeople.
I actually thought that Dr. Jones’s answer was pretty good. IMO, he clearly recognized the game at hand and responded in an accurate and understandable way. Of course, understanding was not part of the agenda of many who quoted him. But the fact that they by and large resorted to distorting his actual words–most often by omitting “statistically”–is a bit telling.
Though the answers given so far are quite good, and would take a bit more twisting yet than Dr. Jones’s, in the end there’s no way to construct a sentence that can’t be turned into a lie somehow.
Say something like “It’s statistically significant at the 90% level”, then let the interviewer respond.
“The last thirty years have seen dramatic, statistically significant warming. There is no indication that the warming of the last thirty years ever slowed or stopped.”
(Critics might say I haven’t answered the question. Quite right — I haven’t. It’s a loaded question, intended to deceive the readers of the interview. The only way to deal with such questions is to do what politicians do; answer the question you want, the hell with the question you got.)
“No I disagree. There is very strong evidence of warming since 1995 and it is statistically significant at the 10% false-positive rate, which is an appropriate criterion given the low statistical power associated with such a short time period”
or, assuming the one-tailed p-value is <0.05
"No I disagree. There is very strong evidence of warming since 1995 and it is statistically significant at the conventional 5% false-positive rate, given the null hypothesis of no warming."
IMO, given the loaded question, a better response would have been to state categorically that it was the wrong question to be asking. And then I would have shown that there is a statistically significant warming trend and it is caused by human actions (quoting the appropriate stats).
In messaging, it’s important to maintain control of the message and not fall for trick questions designed solely to deceive. (Eg Answer yes or no, have you stopped beating your wife!)
I’d have used an approach similar to Easterling and Wehner href=”http:dx.doi.org/10.1029/2009GL037810″>Easterling and Wehner and shown that not only that such short term cooling has happened before during a period of long term warming, but that they also appear in GCM output. I’d then go on to explain what “statistically significant” means, including that not being statistically significant means only that there isn’t enough evidence to rule out the null hypothesis, nothing more, so a lack of statistically significant warming doesn’t mean “no warming”. I’d also point out that an hypothesis being significant at the 95% level of significance *does not* mean that we are 95% confident that it is true (p-values are not a well calibrated measure of evidence) and the 95% level of significance is purely arbitrary (if having the backing of much tradition).
probably say “we can’t measure if the trend was up or down over that short of a period… [climate is a long-term trend, blah, blah, blah] …for every year prior to 1995 going to back to 1970 we know that the trend to this year is positive, and there is no reason to doubt that the current trend will still be positive when we’ve collected enough future data. based on statistics, however, we can never say conclusively what the trend has been over only the past few years.”
that could still be a bit tighter, but i think it improves on his answer….
one crucial thing is to not answer the question exactly as asked. if you’ve ever watched a politician answer questions they hardly ever do. you aren’t obligated to answer the exactly question — something that scientists probably don’t intuitively get because they would get marked down on a test if they’d tried that in school.
analogies are also good, but i can’t think of a good one offhand…
“analogies are also good”
Try the tide coming in on a beach during a rough sea.
Many folk will have experienced trying to work out if the tide is coming in or not and know that the rougher the sea the longer you have to wait before you can be sure.
The data indicates that, for the past 14 years, the trend in warming has continued upwards as it has for much of the past century. However, we use a full 15 year time frame to determine statistical significance and, as we haven’t yet reached that 15 years, I can’t actually answer the question as it has been posed.
Argh, HTML horribleness. Can a moderator delete the previous version?
“Let me be clear about what statistical significance means. If you look at only the data since 1993, our “best guess” of the rate of warming is [X]. The 95% confidence limits range from [A] to [B]. So, while we cannot absolutely rule out zero warming with such a small sample, it is equally likely that the true warming is even worse than the “best guess”. Looking at more years of data allows us to be more precise about the rate of warming. When we do that, it is statistically significant.”
You can go beat up Eduardo if you like: http://klimazwiebel.blogspot.com/2011/01/strange-decade.html
Well I, like many others here, have had to re-answer Jones’s ‘significant warming since 1995’ question many times since in response to people who claim that it meant ‘no warming since 1995’. At the same time, I have also tried to point out that my assessment (for what it’s worth) is that Phil Jones is a decent, honest and thoroughly competent scientist, but like many such is not very ‘media savvy’.
So, this is how I think he should have responded to the ‘significant warming since 1995’ question posed by the BBC:
“Well, the measured surface temperatures of the Earth certainly have increased since 1995 – of that there is no doubt. But there is a small chance (just over five per cent) that that observed warming may be a short-term natural variation, which could have occurred even if there was no general long-term warming trend. So, ‘yes’, the Earth has warmed since 1995, but ‘no’ we can’t quite say that that warming signifies a long-term warming trend – the fifteen years since 1995 is too short to be sure.”
And this was my attempt, some time ago, at an explanation of the ‘no significant warming since 1995’ meme (on the UK Daily Telegraph, after one of Christopher Booker’s articles, I think):
“In effect Jones was asked two questions on this subject, namely “1. Has there been a warming trend since 1995 to the present, and 2. If so, is that warming trend statistically significant?”
The reply he gave made clear that,
1. YES, there has been a warming trend since 1995 and
2. NO, that trend is not (quite) statistically significant.
The situation is analogous to that of a rough sea encroaching onto a beach during a rising tide. The steady rise of the tide is analogous to the steady increase in temperature – the long-term trend. We know that the actual height of the sea on the beach goes up and down as successive waves come in. But the overall long-term effect is of an increase in height of the sea on the beach. On an annual basis, the change in average global temperature due to natural variability (El Nino/La Nina, for example) are some ten times greater (c. 0.2degC) than that due to the long-term trend (c.0.02degC). That is why there is not a steady increase in temperature.
The statement that this warming was (marginally) not statistically significant means that it is just possible (a chance of marginally more than 5%) that such an increase in temperature COULD have come about EVEN IF the long term trend was zero (ie even if neither CO2 nor any other factor was causing a long-term warming). That emphatically does not mean that the long-term trend actually was zero. The reason for the long time (not less than about fifteen years) for the temperature rise to become statistically significant is precisely because of those highly erratic global average temperatures, analogous to the roughness of the sea. To use the tide analogy again: with a flat calm it is easy to quickly determine if the tide is rising. If the sea is rough it takes much longer – that does not mean that the sea is NOT rising, it just means it takes longer for us to be sure that it is.
The actual data, using the HADCRUT3 series to which Jones (and therefore you) referred shows the following:
The five year temperature anomaly centred on 1995 was 0.208degC.
The five year temperature for the last five years (centred on 2007) was 0.414degC.
Ie, the last five years have on average been 0.206C warmer than the five year period centred on 1995.
That’s a good question. Let me answer it thus: “Have you stopped beating your wife yet?” The WMO standard period for putting such meteorological data into a changing climate context is 30 years…
The thing that makes the question disingenuous (and therefore Lindzen and Motl no longer scientists) is that it excises 15 years out of their context in the temperature record–a record that we know can be dominated by noise over such short time periods. Science is about teasing the truth out of complexity, so the only way to answer the question–even scientifically–is to place it back in it’s proper context. I think that the proper answer would be to point to the record and say that we’d had about 20 years of statistically significant warming up to 1995, that there is no statistically significant change in trend and no evidence the physics had changed. Point out that the temperature record is noisy, but that what has changed in recent decades is the fact that greenhouse warming has become nearly commensurate with the noise, bringing the trend to the fore.
Jones was naive. The BBC were reduced to useful idiots and Lindzen and Motl revealed that they were utterly dishonest. The warming trend was an established fact. If Lindzen and Motl were proposing the trend–and by implication the science–had changed, they were under obligation as scientists to vet that contention among their fellow scientists before bringing it before a lay audience. Their failure to do so constitutes gross scientific misconduct.
Alternatively “has the global warming from 1995 to the present been statistically-significant?”
Near enough, near enough.
“No, if you choose a small enough sample size, nothing is significant. You need 30 years to pick out a climate trend.”
This answer gets my vote.
And maybe add, “the data is perfectly consistent with a continuation of the (significant) 30 year trend. There is no statistically meaningful evidence that warming has slowed down or stopped.”
This gets back to my comment on the last thread, but people suggesting answers like “Yes at the 90% level” need to imagine a world where the next such reported asks the next Phil Jones “Do you agree that over the last 7 years there has been no statistically significant warming?” At some point the honest answer is no, the important thing is making it clear that that is not the same as saying it hasn’t warmed.
“I am happy that you asked this question, it’s an important one… now let me slightly reformulate it for you… what you meant to ask was…”
“The trend in the measured global temperature since 1995 has been upwards but, yes, that period is just too short for the increase to be statistically significant.”
Put the emphasis on the fact that it’s the period being too short that is the problem, not any lack of warming.
If we only had data temperature since 1985 then we would only be 93% sure that the World is warming. However we have considerably more data that so our certianty that the World is warming is considerably higher than the 95% significance level that scientist traditionally strive for.
Phrases such as “93% sure that the world is warming” should be avoided in connection with statistical hypothesis tests. The p-value is not the probability that the null hypothesis is true and 1-p is not the probability that the alternative hypothesis is true.
I think I would start with a statement and continue from there: “Because of year-to-year variation, climatologists are not comfortable making conclusions from time periods of less than 30 years. Even so, global temperature has increased over the last 14(?) years and this increase is almost statistically significant; if the 2010 temperature is anything like that of the last few years, the increase since 1995 will be statistically significant.”
First of all, avoid any reply which begins “Yes…” because that is the only part which will be reported. From the perspective of a layman I think the following would be quite effective –
Firstly, I should point out that the question of whether there has been observed warming over a given period is separate from the question of statistical significance.
There has indeed been obserbed warming of about 1.2C per decade since 1995 but that period is too short to establish a statistically significant trend at the 95% confidence level. It we extend the time frame to 1994 or reduce the confidence level to 90% we do get warming which is statistically significant.
I’m disappointed to read all the comments that try to frame an answer to the question as asked. That’s a no-no with trick questions. Forget 1995 (or 1998) as a starting point – that’s just fodder for deniers and gives a false impression of the seriousness of what’s happening. You’ve got to reframe the question and provide a statement that’s relevant to the topic at hand (ie global warming is real and happening now and humans are the cause). It’s no wonder the correct information takes so long to penetrate all skulls. Take control of the interview, don’t be a wuss. (Not saying Prof Jones is a wuss – far from it, he has shown phenomenal courage. Unfortunately he’s just not media savvy.)
(As an aside, the stats classes I attended were eons ago, but it was drilled into us at the time that we shouldn’t just keep dropping the confidence level in an attempt to get ‘statistical significance’. Does that still hold?)
Exactly. Jones should not have answered the question as asked.
Q: Do you agree that from 1995 to the present there has been no statistically-significant global warming
A: The trend since 1995 is 0.12 C / decade. This is perfectly in line with what modern climate science predicts.
Perhaps the proper way to answer the question is to highlight its fundamental dishonesty:
“Hmm, I wonder why you might have chosen 1995? The standard period for judging climatic trends is 30 years. However, even if we were to look at half that period going back to 1994, the trend is still statistically significant. One of the surest ways to fool yourself is to keep dividing your dataset until it gives you a different answer. Now, there will be those who would ask, ‘Well, what if the trend changes in the middle of a 30 year period,’ and the honest answer is that we’ll know in another 15 years. And there is simply no reason to suspect that the trend that we’ve seen for the past 30 years has changed.”
Jones should have pointed out that those who submitted the question were splitting statistical hairs
Jones should have prefaced his answer with a question, “May I have your first born child if the denialist echo chamber misrepresents my answer?”
Jones also says that he’s 100% confident that the world is warming and that he supports the IPCC position, that there’s strong evidence of a substantial human component at its cause. And on the specific question of significance of warming since 1995 he also says that while it fails the the very high 95% test, it fails ‘only just’.
When push comes to shove the anti-science brigade petitioned the BBC to set up Jone’s with a loaded question – which he answered fairly – and then they still latch on to Jones speaking candidly about some of the complications inherent in working with the data and spun his words out of context.
I think it’s harsh to suggest that Jones should have answered differently and wishful thinking to suppose that, against such shifty tactics, there are simply better ways of answering.
The problem isn’t with Jones or his answers, but with those who misrepresent and distort his words.
I’d suggest that a better tactic is to petition mainstream media (including the BBC) to do more to communicate good understanding of the issues. Kind of the point of learning & teaching science, and the BBC has role (if not a duty) to play there, is so that folk aren’t so easily hoodwinked by bullshitters.
In the mean time auntie Beeb’s flagship science series, Horizon, has the Royal Society’s new president investigate ‘Science under Attack’ with particular reference to attacks on climate science. Might be worth a watch.
http://www.bbc.co.uk/programmes/b00y4yql
The politician’s mantra is “don’t accept the premise of the question”.
So, you give the trend over the last 30 years (repetition is another useful tactic – not mindless repetition, but staying focussed on what is important) and *then* you can remind the reader that very short intervals *never* yield a statistically significant result. If you want, you can cap it with a reminder that there is no evidence that the trend has changed.
I don’t know if this was posted but it is worth seeing Jones words in the full context of the interview:
http://news.bbc.co.uk/2/hi/8511670.stm
Some good suggestions made above.
I’m not sure how this interview was conducted (in person, via email) nor what is expected by journalists in terms of these kinds of interviews of scientists.
But in a more general sense, there are good reasons that part of the standard advice in consultative sales is not to simply jump to giving the answer that you think the client is asking. Often you’ll misinterpret the question, and/or not satisfactorily answer it. It’s often better to question for clarification first, and – when you are clear(er) on what the “real” question/objection is – try to briefly rephrase it back to the client to check that you both mapping to a common definition. Then answer the question.
Like I said, I’m not sure how this interview was conducted. But rather than trying – in 20/20 hindsight style – how Jones “should” have answered the question, I think the better takeaway lesson would probably be “how to handle” questions like this. We can never be sure that it would have lead to different answer, but I suspect that this little pause/clarification would have been well worth it to him…
Some very nice answers here.
IMO the lessons are:
* Keep the big picture in mind & recognize that if you don’t take the initiative to convey it, it likely won’t come across.
* As andrew adams mentioned above, beware of saying “yes” even as a segue to reframing the q/issue. The “make the interviewer look good” guideline emphatically does not apply anymore, since it’ll give the audience – and the interviewer – the impression that he *is* good, & that his views are non-fringe, even when this isn’t the reality; and given the importance of this issue…
(Gavin’s Pussycat’s “I am _happy_ that you asked this question, it’s an important one… now let me slightly reformulate it for you… what you meant to ask was” is as far as you should go, IMO. )
* “Perhaps the proper way to answer the question is to highlight its fundamental dishonesty” – amen to Ray. You need to make it clear to the audience, what’s an important Q vs. what’s a misleading one.
* And beware of saying “That’s a good question” for Qs that *aren’t* good at communicating the big picture, since again, it’ll make the interviewer think he’s on the right track, re communicating to the public.
Also, not specifically for Phil Jones, but for any climate communicator, clue your audience in to your network of trust, & what are reliable sources for further information. This should be done anytime you talk in public, to help said public know where they can get good info.
Anna, in my world, I’ve been harping on the need to use reliable sources for the longest time.
It’s kind of funny and a little bit sad to watch people continue to pull up “final nail” articles from the conservative press.
My response would be along the lines of John Cook. My understanding was that Jones had time to form a response, which means being able to:
– Define in layperson terms what “statistically significant” means.
– Mention that shorter time periods have less chance of reaching statistical significance (Jones covered this).
– Mention that there are multiple measures of global warming, including other surface data sets: GISS, which I think shows statistical significance over that period and NCDC, the independent satellite record (RSS, UAH), ocean heat content at various levels and global glacier depletion, and to a lesser extent sea level rise and rapid Arctic sea ice depletion. It’s extraordinarily unlikely that they would all be off in the same direction.
– Conclude by saying that asking for statistical significance of a single data set is not a very meaningful question. For HadCrut, the trend since 1995 falls just short of the 95% statistical confidence level, reaching xx%. If this was the only data set, we could say the small chance of the true trend being zero is also equal to the chance of the trend being double of that observed.
“It has warmed, it continues to warm, we just can’t see the warming precisely in a statistical way, but it hasn’t statistically cooled either.“
Or…
“No, but there is no statistically significant levelling in temperature, and there is no statistically cooling, either. Come back in a year and I’ll have a proper answer for you.”
Or…
“Nobody can answer that question yet, but it has since 1994.”
I’d say:
1. That’s a trick question.
2. Real trends can easily be statistically insignificant when we don’t consider enough data.
3. To test trends like global warming, we need more than 15 years– preferrably 30 or so.
4. But even warming since 1994 passes the standard test for statistical significance.
I think each of these is short and clear enough to be hard to distort, and together they reverse the advantage that the cleverly (and dishonestly) framed question gives to the denier position.
There are multiple periods within the time that humans began experimenting unintentionally with our climate, that we would have to say there was no statistically significant warming. Yet looking back now, we can clearly see the long-term trend as a result of our unmitigated experiment is to warm the planet. Pointing out that the last 14 years are not statistically significant, when the last decade is the hottest decade on record, when the indicators of a warming climate such as declining sea ice, receding glaciers, rising sea level, and the poleward migration of species, would be missing the forest for the trees.
I don’t have alternate answers for Dr. Jones, but I recall that I got a denier version of it in email almost at once, then found the real interview online, and was very frustrated that Jones did not go into more explanation of several questions. Many of the questions were clearly loaded.
A question: He was answering about the HADCRUTemp data. Would GISSTemp have been significant?
[Response: Yes.]
It’s worth pointing out that Jones is on record as saying that GISTemp is crap compared to HADCrut.
Typical in professional fields, but it would’ve been better if he’d been willing to say “the answer depends on which dataset you choose”,
[Response: Do you have a reference for that statement?]
I think dho takes it a notch too far. I apologize for the link, but here’s the relevant e-mail:
http://eastangliaemails.com/emails.php?eid=1042&filename=1254850534.txt
Context may be everything (and he doesn’t say “crap”):
See those emails:
Well, its clear that not everyone on his side of the pond agrees, at least when it comes to capturing the global trend in temperature. In case anyone missed it — from a little over a year ago:
How about this:
=========
All the evidence indicates that warming has continued, it hasn’t stopped or even slowed down. But if you limit yourself to a short enough time span (and for climate, 15 years is a small time span), then you don’t have enough data to show it with statistical significance. So it’s true that the warming since 1995 isn’t “statistically significant” even though it has continued unabated.
It’s like asking whether observing just two people, one smoker and one non-smoker, gives you statistically significant evidence of a relationship between smoking and lung cancer. Of course not — just two people isn’t enough data to make a statistically significant conclusion. But that doesn’t mean there’s no relationship between cigarettes and lung cancer. We all know that there is, because we have a lot more data than just two people.
It’s a common trick of those who deny the reality of global warming, to ignore most of the evidence and focus on such a small amount of data that we lose statistical significance. Pity, that.
=========
Note that it starts by asserting the truth. Then it explains why the lack of statistical significance is due to the paucity of data, not due to the cessation of the trend. It closes by exposing the tricksy nature of denialist propaganda.
nice answer!
I don’t see a whole lot wrong with Jones’ reply. He might have added this from above:
“All the evidence indicates that warming has continued, it hasn’t stopped or even slowed down. But if you limit yourself to a short enough time span (and for climate, 15 years is a small time span), then you don’t have enough data to show it with statistical significance. So it’s true that the warming since 1995 isn’t “statistically significant” even though it has continued unabated.”
Just this much would have clarified his remarks sufficiently for the public and would have cut off the deniers opportunities of making nothing out of nothing on this point and making it appear like something.
We are traveling a bumpy road into the future, and the only seats we have are on the back of the wagon; we see only where we’ve been for sure.
If we only look down at the near ground as we are bumping along, we can hardly say whether we’re on a longterm slope. We are going up and down with every bump. So we take the longer view, in the only direction we can — backward — and we have to look back a ways to tell the bumps in the road from the trend in the slope.
You’re asking me to look at the near ground. If you want that answer, that answer is — even as bumpy as it is year on year, looking only at the most recent fifteen years, it’s upwards of 9o percent likely we’re on an upward slope. And that upward slope is the same slope we’ve been able to see looking, properly, at longer time spans picked anywhere farther back down the road.
If we knew the future we’d tell you. What we can tell you is how likely we think — given how bumpy the road is — that the road we’re on is “very likely” uphill. How likely?
I wouldn’t bet against it…..
Nice image, Hank.
Outstanding. My favorite so far.
Thanks, Eric!
Both Tamino and Hank – I like these analogies a lot. I suppose the next question would be to ask, had Jones used one or other of the good answers people have put here, then… how would your favourite cherry-picking skeptic twist the answer to provide the next Daily Fail headline?
“Roberts says ‘we can hardly say whether we’re on a longterm slope’!!!”
“Tamino says ‘it’s true that the warming since 1995 isn’t statistically significant!!!”
The problem is often not providing a better or worse answer to the question, but finding a way in which your words can’t be twisted by the denial echo chamber – and that as far as I can see is very hard.
I’m curious, now that there have been a number of responses, which do you think is the best?
I’ll vote for Hank Roberts.
Of course, you can still submit more answers!
Hank’s is the archetypical contest entry that, once posted, all others pale besides.
Like a shaft of gold when all else is dark.
The Yooper
John and Tamino
Dear oh dear
Please please could Tamino blog on the interpretation of confidence intervals in classical statistics. I just read John’s comment “When scientists say ‘statistically significant’, they mean there’s a 95% chance of something happening.” John you should know better. This is really not the correct interpretation of a confidence interval. The fact that the correct interpretation is hard to explain isn’t really an excuse for giving the wrong interpretation.
I always end up going along the lines of “If you had 100 time series with the same properties as that one, but not necessarily identical values, and you did the same analysis on each one, then in 95% of these analyses, the trend would be positive, and in 5% it would be zero or negative”. I then go off on one about how in classical statistics the data are fixed and the parameters are random (with Bayesian being the converse), so how if the data are fixed can something have a 95% chance of being true. People’s eyes obviously glaze over before this point.
I realise that my explanation is not ideal, but it is not incorrect. It is much closer than this whole “95% chance of something happening” interpretation which is a travesty!
Any comments?
State the truth but don’t answer the question.
“The data since 1995 alone shows that it is more than ten times more likely to be warming than cooling”
A likelihood ratio might be a better way of communicating the evidence for warming to the general public than a standard Fisherian hypothesis test (which doesn’t show that warming is x times more likely than cooling for any x).
Any discussion of “statistical significance” for the public needs to start with an explanation of what the term means, something like:
“‘Statistical significance’ is a somewhat unfortunate bit of technical statistical jargon, because it means something rather different from what it sounds like. In statistics, when dealing with a measurement that has some degree of random noise or error, we like to quantify how likely it is to make a mistake–to be misled by noise and think that something has changed when it really hasn’t. So you might 10% chance of being in error, or 6%, or 5%, or 1%. But as a handy benchmark, we draw an arbitrary line at the 5% level and call that “statistically significant.” It doesn’t have anything to do with whether the effect is big or large or even whether it is important. So if you have 5.01% chance of being wrong, we say that it is not statistically significant, but if you have 4.99% chance of being wrong, we say that it is, even though there’s not really much practical difference. When measuring a change over time, your ability to distinguish a real change over noise increases a you look over a longer interval of time, and it is always possible to find a time interval so short that it is hard to distinguish a real change from the noise. Given the noise from the day to day and place to place weather variation, the amount of time that would required to reliably detect the increase in temperature expected from physical theory with no more than 5% chance of being wrong is a bit over 15 years. So the answer to your question is “almost, but not quite.” The chance that our measurement of a warming trend is mistaken, using just the measurements since 1995, is just a hair over 5%. Of course, if we go back even a few more years, the chance of the warming trend being an error falls well below that arbitrary 5% benchmark.”
The p-value is not the probability that the null hypothesis is correct, so it would not be true to say that there is a 5% chance of being wrong for a result that achieves statistical significance at the 95% level.
The suggestions are too technical and too long winded.
Ask a question back.
The temperature record is up and down like hiking. If you have been travelling on the flat for the last 15minutes, how do you know you are heading towards the summit or the valley?
Answer: you don’t know, until you climb a tree and look back.
Reference has been made to Lindzen (and Motl) being the source of the ‘no significant warming since 1995’ question to Phil Jones, and how this has misled many into thinking that this meant ‘no warming since 1995’.
I had always assumed that Lindzen knew full well that there HAD been warming since 1995 and understood the meaning of statistical significance.
But then I read this, written by Lindzen, referring to a paper by Smith et al (2007)
” they speculated that natural internal variability might step aside in 2009, allowing warming to resume. Resume? Thus, the fact that warming has ceased for the past fourteen years is acknowledged.”
see: “http://wattsupwiththat.com/2011/01/17/richard-lindzen-a-case-against-precipitous-climate-action/
I must confess, I am rather reminded of some splendid lines from Jorge Luis Borges:
“The greatest wizard (Novalis writes memorably) would the one who bewitched himself to the point of accepting his own phantasmagorias as autonomous apparitions. Would not that be our case.”
Aye, Dr Lindzen, great wizard, in your case I think it is.
In response to the BBC interviewer question (which was this: B – “Do you agree that from 1995 to the present there has been no statistically-significant global warming”) Jones might have concluded his reply with this:
Try to find a link to Open Mind Tamino “How Long” December 15, 2009 and Open Mind Tamino “Riddle Me This” Dec 7, 2009 and read those posts. They’ll tell you pretty much everything you have to know about warming trends and their statistical significance.
I’d probably go for something like Deech or others pointing out first that the period is arbitrary and too short. Pointing out that the period was probably picked for that reason would likely annoy the interviewer. Not that a shot aimed at the interviewer wouldn’t be well deserved – but they probably get a lot more say in the editing than the interviewee and isn’t most of the problem with Jones answer a result of editing?
Tamino, your response was okay except that by the time it was quoted elsewhere it would have become “it’s true that the warming since 1995 isn’t “statistically significant””. Constant calculation of wording to prevent selected bits to be taken out of context is essential and probably tiresome. The problem is largely a result of the media’s search for controversial sound bites and lack of interest in accuracy or desire to inform.
PJKar | January 27, 2011 at 4:55 pm said ……
“Try to find a link to Open Mind Tamino “How Long” December 15, 2009 and Open Mind Tamino “Riddle Me This” Dec 7, 2009 and read those posts. They’ll tell you pretty much everything you have to know about warming trends and their statistical significance.”
I’m very interested in reading these Open Mind Tamino posts. I’ve looked and can’t find an active link to them. Any suggestions on how I might find them?
Thanks,
Jack
[Response: Follow the link to Open Mind Archive on Skeptical Science in the right-hand column (or just click that link).]
Jack,
My remarks in the post come off as a bit glib which was kind of half intended given the slant of the the BBC’s exchange with Jones. Those are two great posts on on the statistical analysis of climate warming trends but there is more.
If you are interested in the multiple regression of temperature data try this set of posts. They form an excellent presentation on the subject:
ttp://tamino.wordpress.com/2010/12/16/comparing-temperature-data-sets/
and you could add this one on trend analysis and statistical significance:
If you need more on PCA there are is a set of 4 or 5 posts in the archive that cover the subject in depth.
For anyone interested in climate science and practical statistical analysis it’s well worth the time to peruse the archives to see what what an amazing resource this site is on those subjects.
@PJKar
Thanks, PJKar. I didn’t take your comments as glib. I’m interested in the multiple regression analysis too and I appreciate your pointing me to those posts.
Jack
(and thanks to the moderator for the “wayback machine” link)