Global Warming Basics: Signal and Noise — and Trend

Even when climate is constant, unchanging, the weather is not.

Temperature is one aspect of weather (and therefore of climate), so it’s in constant flux, whether we’re looking at a single location or an average over the whole globe. The changes we observe, whether of ever-changing temperature or any other weather variable, can be divided into two broad categories: signal and noise.


To understand how they differ, let’s begin with an imperfect but insightful definition of the difference between climate and weather:


Climate is what on an average we may expect,
weather is what we actually get.
— Andrew John Herbertson

By this definition, climate is the signal; it’s what we expect to happen on average. The fluctuations that distinguish weather from climate make up the noise. The weather itself is the combination of “what on an average we may expect” (the signal) and those fluctuations (the noise), so it’s “what we actually get.”

If the long-term average for a given day’s high temperature is 60F (degrees Fahrenheit), that’s the signal. If the actual temperature on that day turns out to be 65F, that’s the weather. The difference — an extra 5F of warmth — is the noise.

The expectation we’re talking about is “over the long term.” If we say the climate is for today’s high to be 60F, that’s what we expect for today’s date over the long haul. But the weather forecast for today might be for a high of 65F, because weather forecasts try to anticipate the noise.

Noise is essentially the random part of the weather. You might be surprised to learn that something random may still be, at least in part, somewhat predictable. It depends on what kind of noise it is, which leads to another fact which can be quite a surprise — that there’s more than one kind of noise. But that is a story for another day.

For a clearer idea of how signal and noise differ, I’ll rely on the old adage that a picture is worth a thousand words.

KREMSMUENSTER

Kremsmuenster, Austria has been a center of learning and of science since it was founded around Kremsmuenster Abbey in the year 777. It’s also the location of a particularly long and high-quality temperature record. Being in Europe, their temperature data are in C (degrees Celsius) rather than F (degrees Fahrenheit).

Here’s the daily high temperature at Kremsmuenster for each day of the two-year period from January 1st 1968 through December 31st 1969:

krems_6870

It certainly fluctuates a lot from day to day, giving the distinct impression of randomness (noise) being part of the temperature. But in addition, there seems to be a pattern, one which repeats consistently: a cycle of seasons, with temperature colder in winter, hotter in summer.

That pattern isn’t followed perfectly; one year is not simply a repeat of the preceding. This is clear if we compare two different years, say plotting temperture by day of the year with 1968 in blue and 1969 in red:

compare_2yr

Although they seem to be following the same overall pattern, the differences can be substantial, as much as 15C (which is 27F) from one year to the next for the same date.

While the pattern isn’t followed perfectly, it is followed consistently. This is clear if we look at more than just two years. Let’s look at the 30-year period from 1940 through 1969:

krems_4070

The cycle of the seasons is evident throughout. That it follows the one-year cycle consistently is obvious by plotting all thirty years by day of the year:

krems_4070_cyc

Yes, colder in winter and hotter in summer is not an accident, nor is it random — it’s predictable. It’s what enables us to have at least some idea what the temperature will be on a given date, years in advance.

The cycle of the seasons is the signal. The fluctuations about that cycle are the noise. That last graph gives us a pretty good picture of both, showing what we expect on average for each day of the year, as well as some idea of how much temperature fluctuates about that expectation. And that’s what climate really is; a better definition is this:


Climate is the mean and variation of weather over long periods of time (typically, at least 30 years).

The mean is the mathematical term for “what on an average we expect.” The variation is the essential nature of the fluctuations about that mean (and is the part of climate left out of Herbertson’s definition). To specify the climate you have to describe the mean (the signal), saying what it is you expect it to be, and you have to describe the variation (the noise), for instance how big the fluctuations are likely to be. To specify climate completely, you have to do so not just for temperature, but a lot of other things too, such as pressure, humidity, precipitation, wind, etc.

ANOMALY

The signal we’ve described so far is “merely” the seasonal cycle. We can quantify it in many ways; I won’t burden you with complex mathematics, I’ll just show you the result of one (very good) way of doing so:

krems_4070_season

The red line shows “what on an average we may expect” based solely on the cycle of the seasons. If, for each day, we subtract that from “what we actually get,” we’ll have transformed temperature into something called temperature anomaly, which just might give us a picture of the noise alone, because we just might have removed the signal. It looks like this:

krems_4070_anom

There certainly is plenty of fluctuation. But overall, it doesn’t seem to be going anywhere — just fluctuating. It’s at least plausible that the temperature anomaly is only noise, that there’s no signal left, in which case that seasonal cycle really is the climate.

SLOWING IT DOWN

Let’s compute the average temperature anomaly for each month of each of the 30 years:

krems4070_monthlyanom

The monthly averages also wiggle around a lot. That’s no suprise, because when you average noise you get — noise.

The noise you get from averaging tends to be smaller than the noise you started with, something that’s clear from the fact that the monthly average anomaly values never get as high as +10 or as low as -10, while the daily anomaly values ranged from well above +10 down to below -15.

It might look as though there’s some pattern to the monthly averages, maybe even some signal present (besides the cycle of the seasons, which we’ve removed). But that’s not the case. This isn’t the simplest type of noise, instead it’s a kind that can easily give the false impression of a signal. To distinguish signal from noise requires some (sometimes subtle) mathematical analysis. I won’t burden you with that either, but I’ve done the analysis and during this time span, from 1940 through 1969, there’s no real evidence that temperature anomaly in Kremsmuenster is anything but noise, so the temperature itself is a seasonal cycle plus noise.

The conclusion is that during this 30-year span, the climate in Kremsmuenster was stable — at least as far as temperature is concerned.

A LONGER VIEW

Let’s average temperature anomaly over each month, but instead of doing so only from 1940 through 1969, let’s do it for the entire Kremsmuenster record, from 1876 through 2015:

krems_monthanom

Now there’s an even stronger impression of some pattern. It really looks like the monthly anomalies have done more than just fluctuate due to noise. In fact the anomalies seem to be higher recently — at least, on an average — than they used to be.

If we average temperature anomaly over each full year rather than each month, that impression is even stronger:

krems_annualanom

The visual evidence of some pattern, some signal, is vivid. But we have to be careful before drawing conclusions; it’s far too easy for the mind to imagine patterns when they’re not real, to see “pictures in the clouds” that look like everything from the Sistine Chapel to the Mona Lisa when we’re really just looking at noise.

That’s where the fancy math comes in. In this case, the pattern turns out to be real; this isn’t just noise, there’s signal there that we haven’t removed, more than just the seasonal cycle.

A better estimate of what that signal is, can be obtained by some other fancy mathematical methods. Here’s the result (shown as a red line) from one of my favorites:

krems_smooth

Now we can plainly see a distinct rise in temperature recently. It’s not a trivial change either; Kremsmuenster temperature has increased by about 3C (5.4F) since right around 1970. Nor is it just noise, not even one of the unusual kinds of noise that gives false impressions, it passes the statistical tests. This is a real change in “what on an average we expect,” i.e. it’s climate change.

Climate-wise, 3C is a lot. The difference in global average temperature between a full-on glacial period and now, between mile-thick ice sheets covering Chicago and warm summers there, is “only” about 5C. The change Kremsmuenster has already seen is more than half that.

And it has happened in less than 50 years. That’s fast.

Averaged over the whole globe, temperature hasn’t increased as much as it has at Kremsmuenster. The global warming so far has only been about 1C, compared to the 3C in this small Austrian town. But in both cases, for Kremsmuenster and for the globe as a whole, the change already has been substantial.

TREND

The upshot is that the temperature signal at Kremsmuenster is more than just the seasonal cycle. That cycle continues, but in addition there has been a rise over the last decades. Both the cycle and that upswing are part of the signal.

There’s a name for the part of the signal that isn’t cyclical; it’s called the trend.

Sometimes when people say “trend” they’re referring to a linear trend. That’s a pattern which follows a straight line. But another, and often more useful, definition is whatever part of the signal isn’t cyclic, and that can often be (as in the case of Kremsmuenster temperature) different from a straight line.

We began by decomposing temperature data into two components: signal and noise. We have now further decomposed the signal into two components: cycles and trends. For temperature, the cycle consists of the coming and going of the seasons. The trend is … global warming.

TREND AND NOISE

There’s a saying, that “just because climate changes, that doesn’t mean we won’t still have weather.” The point is that just because there’s a signal, in particular a trend, that doesn’t mean we won’t still have noise.

The noise makes it harder to see what the trend really is. If one day is hotter or colder than the day before, it’s easy to believe that could just be noise, one of those unpredictable fluctuations from day to day. But many people aren’t aware that such fluctuations exist on longer time scales: from month to month, from year to year, even from decade to decade. The impact of noise is to bring about change in temperature for no apparent reason.

If one year is hotter or colder than the year before, that too might just be noise. But it could also be a combination of year-to-year noise and a trend. And if the time span we’re looking at is short, chances are the trend (if there is one) hasn’t had enough time to accumulate enough change to be noticeable compared to the noise. It takes time for the signal to “rise above the noise.”

That’s one of the tricks global warming deniers use to claim the absence of a trend, or a change in (or even a halt to) global warming. They limit what they show you to a brief span of time. That way, the jumping up and down due to noise (which is still there, even when climate changes) can hide the trend.

They usually pick an especially hot time to start, one which is extra hot due to the noise. That often means 1997 or 1998, because that’s when a giant el Niño made temperature soar. But el Niño is one of those noise factors, not part of the trend, so the years that followed weren’t as hot, and that made it look like the trend was going down, when really it continued steadily up.

Any time someone shows you a temperature graph starting in 1997 or 1998, you can rest assured that they’re trying to take advantage of the noise to mislead you about the trend. Don’t trust ’em.

As for noise — the fluctuations — making the trend harder to see, Neil DeGrasse Tyson illustrates that in this video:

It’s an excellent description. It’s especially good at revealing the difference between changes that are persistent and give us an idea where we’re heading, and those which are random and really don’t tell us anything about what to expect in the future. That’s what climate is about: what we expect to happen, on average.

My advice is twofold. First, don’t trust those who start their temperature graphs in 1997 or 1998. Second, keep your eye on the man, not the dog.

If you value these essays, please help by donating at Peaseblossom’s Closet.

29 responses to “Global Warming Basics: Signal and Noise — and Trend

  1. Excellent!

  2. Sheldon Walker

    Tamino,
    do you regard yourself as a fair person?

    You said, “Any time someone shows you a temperature graph starting in 1997 or 1998, you can rest assured that they’re trying to take advantage of the noise to mislead you about the trend. Don’t trust ’em.”

    But you didn’t say, “Any time someone shows you a temperature graph starting in a cold year, you can rest assured that they’re trying to take advantage of the noise to mislead you about the trend. Don’t trust ’em.”

    I have heard Alarmists make the comment about not starting a graph in 1998 at least 40 times. I have never once heard them advise against starting a graph with a cold year.

    Do you think that they could be biased?

    [Response: If someone started with a cold year which was cold because of the noise, in order to give a false impression of warming, I’d call them on it. Notice that sentence starts with the biggest little word in the English language: “if.”

    I might start a temperature graph with 1880, or 1850, not *because* it was colder back then but because that’s when the *data* start. I might begin an Arctic sea ice graph with 1979, not because there was so much more sea ice back then but because that’s when the satellite data start.

    I might start a temperature graph with 1970 or 1975, not because it was colder back then but because that’s a time when we can show, with statistical rigor, that there was a meaningful change in the trend. But when you look for the same thing in 1997/1998, using rigorous statistics, it *fails* to show a meaningful change. I’ve even published research about exactly that — so have half a dozen other guys.

    The real dishonesty behind the 1997/1998 thing is that those pushing that idea don’t want you to see what happened before, the context in which the 97/98 monster el Niño caused a spike.

    I’ve been at the forefront of the effort to put the so-called “pause” in its proper context, and subject it to real statistical analysis. But now that 2015 has smashed the record, I’m starting to see people (on other forums) talk about global warming *accelerating* and “taking off.” I’ve reacted by saying out loud that we shouldn’t be making such claims — that too doesn’t pass statistical muster, we don’t really have the evidence yet to talk about acceleration. I’ve even mentioned that if we do so, we’re just falling into the trap so common in statistics, that appearances can too easily be deceiving.

    But I don’t call them dishonest, because it’s a natural mistake. Those who continue to push 1997/1998 I *do* call dishonest, because they’ve been informed — many, many times — that it ain’t right, they’ve been shown the research putting such claims under the microscope, but they persist in sticking to the same deception.

    I don’t consider myself “alarmist,” but I am alarmed. If you want to argue that I overestimate the threat, that’s one thing. But if you want to deny that global warming continues apace, that’s another thing entirely. Denying reality obstructs any meaningful discussion of how severe the danger is, and what we should or should not do about it.]

    • Dear Sheldon (if I may dare to be so familiar),
      I respectfully suggest that you are making two mistakes:
      1) I would fully agree with any criticism you might make of people selectively choosing a data start date to suit their argument (aka “cherry picking”). As Tamino points out, one side of the climate change argument regularly does this. Do you call them out for this? The other side, despite your comment, I have never seen do this. I stand ready to receive your list of occasions when it has been done….
      2) It is unfortunately common human behaviour to believe that the other guy’s mistake allows us to do the same thing back. Do you believe that it is permissible to cherry pick 1997/1998 for a data start year because some “alarmist” (aka realist), somewhere, some time, cherry picked a date that suited them? May I remind you of the common saying “two wrongs do not make a right”. And I am a regular reader of Tamino because he clearly does NOT make this mistake. As a quote from Friedrich Nietzsche says:
      “Beware that, when fighting monsters, you yourself do not become a monster… for when you gaze long into the abyss. The abyss gazes also into you.”

      • Sheldon is one of those “concerned” people. This time he is “concerned” about all those “alarmists” who widely promote disinformation by intentionally picking cool years as a starting point for analysis. A couple of days ago he was concerned enough to inform Sou of some nonissues over on Hot Whopper as well.

      • Ah, well, now that I know that it’s all for my own good, I’m sure I’ll soon feel the proper gratitude for Mr. Walker’s concern. After all, it might not have occurred to me that there were potential symmetries in cherry-picking, given the observational record.

        Come to think of it, it’s a bit of a puzzle, isn’t it?–rather like why there is so little antimatter to be found. You’d naively think that the distribution of matter and antimatter might have been symmetrical, but not so. Luckily, discovering just why cherry-picks overwhelmingly involve the classic warm-biassing “98 case” is unlikely to involve particle accelerators.

    • @Sheldon Walker:
      The main error starting trend calculations in 1997 or 1998 is not that 1997 or 1998 are warm years but that the time since then is too short to establish a robust trend. So the errors are very high and the results are very sensitive to if the starting year is warm or cold.
      So I would not trust anyone who calculates trends for global mean surface temperature from less than 23 years.(*) Regardless if the start year is warm or cold (or average).
      If you chose a sufficient long time span, it will matter not so much if you start the trend in a warm or cold year, as you can check.

      And for a bias in comments not starting in warm years vs. not starting in cold years, you may ask if the bias is not in the comments but in that much more people showing trends starting with warm years that people starting in cold years? How oft see you trends calculated based on too short time periods (less than 23 years) and starting in warm or starting in cold years? Would the conclusion from this trends chance, if the trend would be calculated from a timespan with at least 23 years?

      And for showing temperature graphs, a fair person should also show the whole graph without cropping years at the beginning or the end.

      (*) It may be possible to use a few years less, if the noise from the ENSO is properly removed. But this is only for advanced statistics.

      • An addition: Of course warm year means starting above the trend cold year means starting below the trend line.
        Starting in a warm year does not mean starting above the mean and cold year starting below the mean, which would be a completely wrong labelling of warm and cold years in this context.
        So if one does a trend calculation from 1958 to 2015 he is starting in a warm year because 1958 is above the trend line despite of 1958 is below the 1958 to 2015 average.

  3. Thanks for the tutorial.
    How is a new record high temperature or a new record of low ice extent for a month or a year fit in where a trend is already evident,is it noise? can it be regarded as a continuation of the existing trend or a bit of both?

    [Response: A new record is “what we actually get,” so it’s the combination of signal (i.e. trend) and noise.

    The trend in global temperature right now is about 0.016 deg.C/year. If there were no noise, each year would be that much hotter than the one before, and each year would be a new record.

    But the noise from year to year is about 0.1 deg.C, six times larger, and that’s just the “root-mean-square” noise level, it can easily be larger or smaller (unpredictably, as noise is) and it can be up or down. So to get a new record you usually need the noise to be hot. But if there’s no trend, the chance of the noise being hot enough to set a record is extremely small. As the trend (“what on an average we expect”) creeps upward, the chance of a new record is much greater. That’s why the noise has been hot enough lately to set the record many times — because the trend is creeping upward.

    The record heat of 2015 is the combination of the kind of noise due to what’s called “el Niño,” and the ongoing trend. Record heat makes headlines. The trend foreshadows what to expect.]

    • Thanks for taking the time,these basic tutorials help break down the jargon for us novices

    • can it be regarded as a continuation of the existing trend or a bit of both?

      For what it’s worth, 2014 was both pretty much on trend and a new record warmest year in GISTEMP. This is much more likely if there is a run of below-trend years (2011, 2012, 2013).

      • Why? The known positive autocorrelation in the temp series would seem to me to tend to predict the opposite–i.e., some small tendency towards continuation of the previous low values. Random noise wouldn’t predict anything at all (see Gambler’s Fallacy).

        There are physical reasons to predict a warm 2016, but they have nothing to do with the run of below-trend years.

        Or am I missing something here.

      • Well yes. I was talking about cases where there is new record and it is on trend.

      • Some Climate Etc. comments claimed 2014 was going to be the highest temperature seen in the record for decades. The Stadium Wave was about to latch, etc. For those who religiously believe in flat things, 2015 is a mighty thorn up the chute, and they are doing their best to act like it is not happening.

  4. 1 thought about Sheldon Walkers argument:
    I would object to anyone showing me any “trend” whatsoever based on a 15-year-set of data. It just has no predictive value, given that the noise properties of the system, which we know, don’t change much over time – which is a perfectly reasonable assumption (an assumption nevertheless).

  5. Tamino – in case you aren’t aware, Sheldon Walker has guest posted at WUWT.

  6. The 4th graph, the plot of thirty years of daily temperatures over a year, appears to show a seasonal variation in the noise to signal ratio.

    The vertical scatter of points looks greater in the spring – F/M/A than in the autumn -S/O/N.
    Is this a real difference, is it a local effect or a common feature of weather records?

    [Response: It’s a real difference. I haven’t examined enough records to know how that happens globally, but I have noticed it in the locations I’ve examined in detail.

    Bear in mind also that these are daily high temperatures; the pattern is different for daily mean temperature, where the scatter peaks in January (at Kremsmuenster). Generally, I’ve noticed more variation in winter and less in summer. And, the variation is more “skewed” in winter, more “normal” in summer.

    I’d say the whole thing deserves detailed study. Maybe someday …]

  7. I find that when it comes to discussing climate it is wise to never trust anyone who uses the term “alarmists.”

    • Especially if they capitalize it, as a category or organized group rather than a (rather poor) description.

  8. David B. Benson

    Following the SI conventions, for 20 degrees Celsius, write
    20 °C
    Note carefully the space between the number and the degree sign and the lack of space after it.

    I suppose the same convention is followed for degrees Farenheit but that is not an SI unit.

    From SI abbreviations, 20 C denotes 20 kilocalories, a measure of food energy.

    • This is why for temperature differences I always use K.

    • Inspired me to finally learn how to type ℃. On my new 12-inch macbook, control – command – shift, and then search through a large menu. As a layperson, K makes me nervous. I read a paper the other day that estimated ECS at 3.9 K, so I nervously assumed that was 3.9 kilocalories.

    • David B. Benson

      In the SI system of units the unit is kelvin, abbreviated K, and the degree sign is never used.

  9. Can you say something about the definition of the “cycle of the seasons”(even just the technical term(s) for the method)? How many years are needed to extract the periodic component? What would happen if you were to decompose the signal from 1998 to 2015, where the anomaly is increasing? Would the cycle agree with what you get for the whole period from 1880?

    [Response: Yes it does. Using just the data from 1998 onward (at Kremsmuenster), the cyclic component has an estimated period in the range from 0.9993 to 1.0014 years, so it’s 1 year within the margin of error.

    This is a case where the cycle is big enough that it’s easy. Even if not, linear trends don’t interfere with cyclic variation *much*. Finally, one can allow for the trend when estimating a cycle’s period. Generally, when both cycle and trend are estimated simultaneously, results are much better.

    And for this specific case, we know the physics behind the cyclic behavior. Bear in mind that the *definition* of the year is the average length of the cycle of the seasons. That’s why our calendar is based on the “tropical year” (cycle of the seasons) rather than the “sidereal year” (time for earth to orbit the sun).]

  10. Tamino is correct about when, where and why a certain time-series is begun. In simplest terms, you always start from the earliest data point you have (unless you have good reason not to). Otherwise, a population census of passenger pigeons begun in 1940 would show they never existed.

  11. I visualise the temperature trend a bit like monitoring ones weight – to see if you are losing or gaining weight

    obviously your body weight is subject to short term “noise” i.e. when you last drank/ate, morning or evening, pre or post “ablutions” , pre or post exercise, clothed or naked – the list is long

    all will affect the actual reading of the scales – but the “trend” will be obvious
    sure weigh yourself fully clothed (in an el nino year), then weigh yourself the next day naked – and you can con yourself you are losing weight!!!

    But as Feynman said – “You can’t fool nature”

  12. Would observing the relationship between record high daily temps vs record low daily temps over climatic time scales be another useful way of looking at temp trends?

    • There is already plenty of research on the ‘diurnal range,’ which is the difference between night-time and day time temperatures. Here’s a list of papers on the subject – not up to date.

      https://agwobserver.wordpress.com/2010/11/24/papers-on-diurnal-temperature-range/

      It appears that, globally averaged, nights warmed faster than days from the 1950s, while the diurnal range seems to be stable (or reversed) if the analysis starts from the late 1970s/1980. But it’s been some years since I read a bunch of research on it. The prediction for warming due to GHGs is that nights would warm faster than days.

      I’ve wondered if the apparent stability (possibly reversal of the trend) since ~1980 is a statistically significant result, and put the question to Tamino here some time ago. If you’ve time and interest at some point in the future, Tamino, I’m curious about this global ‘fingerprint’ of greenhouse warming over the last few decades.

    • Meehl et al published on that a few years ago: http://onlinelibrary.wiley.com/doi/10.1029/2009GL040736/abstract