Hurst

When doing trend analysis, one of the most common ways to compensate for autocorrelation in the data is to compute an “effective number” of data points when estimating the uncertainty of the trend value, rather than simply use the raw number of data. To compute the effective number, we begin by computing the number of data points per degree of freedom as

,

where is the autocorrelation at lag . We then divide the number of data points by the number of data points per degree of freedom to get the effective number

.

Then, when computing the uncertainty in our estimated trend rate, we substitute for .

If the random part of the time series is a 1st-order autoregressive (AR1) process, then the autocorrelation coefficients are given by

,

where is the autoregressive parameter. In this case the number of data points per degree of freedom can be estimated as

.

This is why, when estimating the uncertainty in a trend rate of a time series for which the random part is assumed to be AR1, we change the number of data points to the effective number . Such a procedure is probably the most common way of compensating for autoregression in trend analysis of geophysical data.

The original formula is based on the fact that when we estimate the probable error in some parameter (like an average, or a trend rate), we can in fact compensate for autocorrelation by computing a number of data points of freedom per degree of freedom

,

where is some factor (between -1 and +1) which depends on the nature of the parameter we’re estimating.

For most parameters (including averages and linear regression trend rates), the factors are close to 1 at low lags , and if the autoregressive coefficient is not too big, then by the time the factors get much less than 1, their coefficients are very small indeed. Hence we can approximate the sum by simply approximating for all lags . Also, for lags greater than (the limit of the actual sum), the coefficients are quite small, so we can extend the sum to infinity without changing the result very much. Setting all the factors equal to 1, and extending the sum to infinity, this formula becomes the first equation in this post. It’s only an approximation, but the infinite sum with all will be greater than the true sum, so it’s a conservative (i.e., safe) approximation.

We can apply the first equation of this post even if the autocorrelation coefficients do not behave as an AR1 process. They might behave according to a 2nd-order autoregressive (AR2) process, or an autoregressive-moving-average (ARMA) process, or any number of other possibilities, but the original equation can still be applied to give an approximate estimate of the number of data points per degree of freedom. For most random processes (like all ARMA processes), this is no problem. But there’s a class of random processes for which the sum

,

does not converge; it’s infinite. A time series of such a process is said to exhibit long-range dependence and is called an LRD time series. For an AR1 (or any ARMA) time series the sum will converge (i.e., be finite), but for an LRD process it’s infinite.

This of course will wreak havoc with application of the original equation in this post! If the number of data points per degree of freedom is infinite, then the “effective number” of data points is zero and the uncertainty in our estimated parameter is infinite. That’s not a good thing, if we want to estimate the parameter with any degree of reality.

One of the first to investigate the nature of LRD was Harold Edwin Hurst, who noted long-range dependence in hydrology data related to the flow of the Nile river. He noted that the autocorrelation coefficients didn’t diminish as rapidly as they do for most random processes. The autocorrelation coefficients should diminish at greater lags ; the “degree” of divergence of their sum depends on just how rapidly (perhaps I should say, how slowly) they diminish. For an AR1 process they go as , i.e., they decay exponentially; that’s plenty fast enough decay for the sum to be finite. But for LRD processes we can often model the decay as

.

I’ve used the symbol to indicate “assymptotically equal to,” i.e., as lag gets very large the equation becomes a better and better approximation. The constant is just some constant, and the factor is the power to which we raise the lag to get the autocorrelation coefficient. For the autocorrelations to be realistic (i.e., to diminish at greater lags) the parameter must be greater than zero.

When is greater than 1, the sum converges and the time series does not show long-range dependence. When is exactly 1, the sum diverges logarithmically. When is less than 1, it diverges even faster. The decay of the autocorrelation coefficients can also be characterized by the Hurst parameter

.

Since LRD time series are characterized by between 0 and 1, they’re characterized by Hurst parameter between 1/2 and 1.

Is this the death-knell for estimating trend rates using linear regression for LRD time series? Not at all! Remember that the first equation in this post is only an approximation. The correct equation involves a finite sum rather than an infinite one, and involves factors depending on the parameter we’re estimating, which are less than 1. For example, if we’re estimating the average of the time series, then the factors are given by

.

This has some interesting consequences, particularly the fact that if we estimate the average using a number of data points, the variance (square of the probable error) of our estimated average decreases slowly. When there’s no autocorrelation, the variance of an average over data points tends to be proportional to , i.e., doubling the number of data cuts the variance in half (so the probable error is what it was before). But for a time series with LRD, the variance of an average over data points decreases as , and for less than 1 (the condition for LRD) this means our uncertainty declines more slowly.

We can take our sample of data points and split it into blocks of data points each, computing the average for each point block. We can then measure the variance of these values as an estimate of the variance inherent in a point average of the time series. This variance should be proportional to . So if we study how the variance depends on the number of data points per block, we can get an estimate of the parameter . This then enables us to estimate the Hurst parameter . In fact this is one method for estimating the Hurst parameter (the “aggregated variance” method).

Of course, we want the blocks to be reasonably large (to generate as precise estimates as possible), and we want as many of them as we can get (for a more precise estimated variance of the averages themselves), and we want to be able to do so over a wide range of values so we can precisely estimate the relationship between the variance of -point averages and . All this means that to get anything like a precise estimate of the Hurst parameter, we need lots of data. This is not unusual in some situations (e.g. telecommunications network traffic) but is next to impossible in other situations (most of science). Hence if we have evidence of LRD in a time series, often one of the most difficult problems is to quantify it with any degree of precision. This emphasizes the difficulty obtaining reliable estimates of the behavior of time series (especially short ones) which exhibit LRD.

As simple as the aggregated-variance method is, and as many other methods as are available, estimates of the Hurst parameter are tricky even with a pure LRD process and lots of data, and are plagued by uncertainty when the time series is anything other than pure. For example, adding AR1 noise to the series will corrupt most estimates, as will adding a trend or a cyclic signal. Even when trends/cycles are estimated and removed, corruption of the estimate of the Hurst parameter remains. To quote from Clegg (2005, arXiv:math/0610756v1)

… measuring the Hurst parameter, even in artificial data, is very hit and miss. In the artificial data with no corrupting noise, some estimators performed very poorly indeed. Confidence intervals given should certainly not be taken at face value (indeed should be considered as next to worthless).

Corrupting noise can affect the measurements badly and different estimators are affected in by different types of noise. In particular, frequency domain estimators (as might be expected) are robust to the addition of sinusoidal noise or a trend. All estimators had problems in some circumstances with the addition of a heavy degree of short-range dependence even though this, in theory, does not change the longrange dependence of the time series.

When considering real data, researchers are advised to use extreme caution. A researcher relying on the results of any single estimator for the Hurst parameter is likely to be drawing false conclusions, no matter how sound the theoretical backing for the estimator in question. While simple filtering techniques are suggested in the literature for improving the performance of Hurst parameter estimation, they had little or no effect on the data analysed in this paper.

One small “saving grace” is that the nature of trend analysis makes it less susceptible to LRD than many other parameters we try to estimate. This is because the factors for an estimated trend rate from linear regression are

.

For very low lags this is close to 1, but for larger lags (for lags greater than about one third of the number of data points) the factor becomes negative. Here’s a plot of the factor , as a function of lag , with data points:

The fact that the factors become negative means that if the autocorrelations at lags greater than about one third of the number of data points are large (as is the case with LRD), they will actually decrease the uncertainty in our estimated trend rate! It also means that increasing the Hurst parameter doesn’t necessarily make an estimated trend more uncertain; when the Hurst parameter exceeds a “critical value” the uncertainty in an estimated trend actually decreases. In fact I’m glad the issue of LRD was raised here, because it’s spurred me to investigate the effect of LRD on estimated trend rates, and how their uncertainty changes with the Hurst parameter.

Of course LRD is still a great difficulty in time series analysis; it makes estimates of even the most fundamental quantities (like the mean value of the time series) far more uncertain, and getting a handle on the characteristics of the LRD (like the Hurst parameter) is always uncertain and calls for very large quantities of data. But LRD is not an insurmountable problem for time series analysis. It calls to mind the old maxim, that every challenge is also an opportunity.