Warts and All

Legend says that when Oliver Cromwell had his portrait painted by Sir Peter Lely (who had done the royal portrait of Charles I of England, and would later do the royal portrait of Charles II), Cromwell so disdained that personal vanity typical of royalty, that he instructed Lely to paint him as he was, “warts and all.”

When it comes to global temperature estimates like that from NASA GISS, we are often assaulted by claims that climate scientists misrepresent the data, concealing the flaws to give a false impression of their usefulness. Such claims are dishonest, because NASA scientists have exposed their data, their methodology, and their computer programs, to the most detailed scrutiny. The whole package is available for download from the internet, and some interested and independent parties have actually reproduced the entire calculation. They’ve even identified a few bugs and made some improvements (especially in how the computer code is written), but nothing so far that affects the essential results. In essence, they’ve confirmed that NASA GISS got it right. NASA’s work has stood the test of the closest inspection, warts and all.

That doesn’t stop those in denial from focusing exclusively on the warts. Recently, Anthony Watts has echoed Roger Pielke Sr. pointing to the inhomogeneities in the temperature record from De Bilt in the Netherlands. The implication (one which they have repeatedly emphasized) is clear: that the entire GHCN (global historical climate network) data set is useless for estimating long-term temperature trends, either locally or globally.

Much effort is expended to adjust the data, in order to improve its representation of long-term temperature change. If data are used unadjusted it’s claimed that they’re utterly useless for estimating long-term trends, but if data are adjusted, those doing the actual work are accused (by those who are unwilling or unable to do so) of fraud, of deliberately imposing adjustments which exaggerate the global warming trend. Damned if you do, damned if you don’t. And those doing the damning point to inconsistencies in rival adjustements, reinforcing their implication that the data are useless. It’s a sentiment we’ve heard many times.

Here’s the raw (GHCN monthly) data for De Bilt:

There’s certainly a lot of variation in the data. There’s also visual evidence of discontinuities, two of which are indicated by the dashed vertical lines. One of the ways such discontinuities can be identified is to compare data to that from nearby locations. De Bilt is at latitude 52.1N, longitude 5.2E. Let’s take all the stations in the GHCN, unadjusted, in a large grid box from 0 to 15 deg. E longitude and from 50 to 60 deg.N latitude, and compute a gridwide average of temperature anomaly. Then let’s subtract the gridwide average from the De Bilt data to see how De Bilt differs from its neighbors in this rather large region. Here’s the result (plotted on the same scale):

This makes the discontinuity quite clear. Evidently the De Bilt data has “warts” — its record is inhomogeneous, which can confound estimates of long-term trend. This is even more evident if we compute 1-year averages of the difference between De Bilt and the gridbox average, and expand the y-axis:

Do such problems make global temperature estimates useless? Certainly not. It’s fool’s wishful thinking to believe that the errors don’t go both ways in random fashion, and that by taking averages of many data records over large regions they won’t, to a large degree, cancel each other out, enhancing the true signal while suppressing the noise. It’s also foolish to believe that adjustments can’t improve the situation, helping to tease the signal out from the noise. Alas, that doesn’t stop fools from claiming that all the warts go the same way (to exaggerate warming) — a belief which is frankly ludicrous — and all adjustments make matters worse.

We can test the idea that when averaged over large areas, the warts tend to cancel while the signal is reinforced. We can compare averages from large regions which are near to each other, but don’t overlap so they’re independent, to see whether or not they give a similar representation of the underlying signal. It’s well established that temperature changes tend to be correlated over large distances, so if the averages correlate strongly we have evidence that they’re capturing signal more than noise — it’s beyond belief that all the “wart factors” from different data records would conspire to create the same long-term changes.

So: let’s compare the gridbox averages from the “De Bilt” grid — using raw data (warts and all) including De Bilt — to those from its neighbor grids. We’ll compare it to the four nearest grid boxes, to the north, south, east, and west. Hence grid zero (with De Bilt) is Lat 50N to 60N Lon 0 to 15E. Its neighbors are Lat 50N to 60N Lon 15E to 30E, Lat 50N to 60N Lon 15W to 0, Lat 40N to 50N Lon 0 to 15E, and Lat 60N to 70N Lon 0 to 15E.

If we compare 1-year averages of gridbox averages, we get this (neighboring gridboxes in black, the central gridbox in red):

The agreement is — how shall one say? — impressive. We can get an even better idea of the long-term trend by comparing 5-year averages:

Impressive indeed. Since NASA GISS only estimates global average temperature from 1880 to the present, let’s take a close-up look at those 5-year averages from 1880 through 2010:

We can also, of course, smooth the data:

And of course we can zoom in on the smoothed estimates from 1880 through 2010:

Impressive indeed.

Of course there are differences! For one thing, these gridboxes are rather large — and we can’t expect such large regions, separated by such large distances (about 1000 km. apart from center to center) to agree perfectly, there really are regional differences in temperature patterns. For another thing, the data are far from perfect. There really are warts — plenty of them — so none of the gridbox averages is perfect.

But it’s more than just foolish, it’s dishonest (to one’s self as well as others) to believe that the GHCN data are so dominated by the warts that they’re useless for estimating long-term temperature trends. If that were true, we wouldn’t see anywhere near as much agreement in neighboring regions. The idea that the agreement is due to data imperfections, or adjustments (and for this comparison there aren’t any), or random fluctuation, quite simply beggars belief. We also wouldn’t see such strong agreement between global temperature estimates from multiple sources (when corrected for exogenous factors):

Why, then, do those in denial cling to such beliefs? It has to do with that warming signal seen in the most recent decades, the most consistent feature of these gridbox averages. The one that makes the record look like a “hockey stick.”