Find Your Best Market to Trade With the Hurst Exponent

After five consecutive years of drought, Northern Californians welcomed the heavy rainfall in the winter of 2016-2017. By February, however, the rain had led many to worry about the integrity of the Lake Oroville Dam. Officials evacuated over 200,000 residents who lived downstream of the dam along the Feather River and engineers opened the emergency spillway. Soon, however, a small crack in the spillway developed into a massive crater, causing the spillway to collapse and the deluge to cut into the side of the hill.

The rapid erosion threatened to destroy the entire reservoir, leading to a 10m wall of water wiping through the communities.

Thankfully, the dam held and the rains subsided, sparing the homes of the evacuated towns below.

As this episode demonstrates, designing sufficiently large reservoirs is a high stakes problem. Get the calculations wrong and thousands of people could lose their homes, or worse. In the mid 20th century, Edwin Harold Hurst, a British hydrologist made a major breakthrough in solving these kinds of problems. He developed what came to be known as the Hurst Exponent, a measure of the auto correlation in a time series. Although it was developed to estimate the volume of water in a river, the Hurst Exponent, as you might expect for a mathematical calculation involving a time series, has found interesting applications in the study of financial markets, especially in the work of Benoit Mandelbrot.

TL;DR

We show you how to calculate the Hurst Exponent and use it to evaluate properties for the S&P 500, Bitcoin, and the USD/CHF pair to see if we can understand the tendencies of these financial series for trading.

What is the Hurst Exponent?

The Hurst exponent can be used to determine whether a time series is tends to move in a single direction (H>0.5), oscillates (H<0.5), or is random (H=0.5).

While Hurst found this relationship to be useful in various natural phenomena such as floods, river discharges, and tree rings, we can also use it to categorize a market as trending or mean reverting. If we know that a market tends to behave in one way or the other, we can try to capture that with appropriate mean reversion or trend following strategies or use it as a filter in your algorithmic trading system.

How do you Calculate the Hurst Exponent?

Calculating the Hurst Exponent requires estimation of the R/S statistic at multiple different time periods, then plotting that against the time periods on a log-log plot and finding the slope. The slope of that line is the Hurst exponent H.

There are multiple ways to estimate the Hurst exponent. As far as I can tell, the rescaled range analysis method is the oldest (see chapter 8 here), but using detrended fluctuation analysis (DFA) is the most common. We'll use the latter because, although it sounds complex, it only requires 3-4 lines of code to get it to run and is about an order of magnitude faster than my rescaled range analysis implementation. Both these approaches are viable, but they differ slightly.

Anyway... let's get to the math so we know what we're doing.

Say we have a price series we call x. Then we can look at the difference between x at different times t and take the difference. The time difference is going to be called or the lag. We want to get the relationship of the standard deviation of the differences across these lags.

We can write our calculation in the following steps:

Choose a range of lags (e.g. 2 to 100).
Calculate the lagged difference between all the points in x.
Calculate the standard deviation for each lag.
Plot the log of the standard deviation against the log of the lags to estimate H.

Mathematically, we have:

$$D_{\tau} = x_{t+\tau} - x_t \quad \forall \tau$$ $$\sigma_{\tau} = \sqrt{\frac{1}{N} \sum(D_{\tau} - \bar{D}_{\tau})^2} \quad \forall \tau$$ $$\sigma_{\tau} = \tau^H \quad \forall \tau$$ $$\textrm{log}(\sigma_\tau) = H \textrm{log}(\tau)$$

Plugging in all of the data into this last equation, we can find the best fit line and we have our value for H!

Let's show how to do this in Python!

Calculating the Hurst Exponent in Python

Start by importing a few packages:

import numpy as np
import matplotib.pyplot as plt

And now for our hurst() function.

def hurst(price, min_lag=2, max_lag=100):
  lags = np.arange(min_lag, max_lag + 1)
  tau = [np.std(np.subtract(price[lag:], price[:-lag])) 
    for lag in lags]
  m = np.polyfit(np.log10(lags), np.log10(tau), 1)
  return m, lags, tau

We can wrap everything into a few, tight lines by using a list comprehension that calculates the standard deviation of all of the lagged differences.

Next, let's test the function on some series that we know the answers to. This will help ensure our calculation is correct.

N = 10000
rand = np.cumsum(np.random.randn(N) + 0.01)
mr = np.cumsum(np.sin(np.linspace(0, N/3*np.pi, N))/2 + 1)
tr = np.cumsum(np.arange(N)/N)

m_rand, lag_rand, rs_rand = hurst(rand)
m_mr, lag_mr, rs_mr = hurst(mr)
m_tr, lag_tr, rs_tr = hurst(tr)

print(f"Hurst(Random):\t{m_rand[0]:.3f}")
print(f"Hurst(MR):\t{m_mr[0]:.3f}")
print(f"Hurst(TR):\t{m_tr[0]:.3f}")

Hurst(Random):	0.499
Hurst(MR):	    0.075
Hurst(TR):	    0.997

Moreover, we can plot these series along with the log-log plot of our lags and statistics.

The plots in the left column are the log-log plots. You can see the standard deviation statistic we're trying to fit on the y-axis. For the random and trending time series, they match up well. The mean reverting one - which is a rapidly oscillating sine wave - doesn't line up at all, but the slope is still positive and less than 0.5. You'll never see a financial time series like this in your life, but it should give you confidence that our function works!

Hurst Exponent for Financial Time Series

Applying Hurst to financial data is just as straightforward. We can apply it to a few, common time series to see what we get.

We'll use USD/CHF, BTC, and SPY just to get a feel for each of these.

Grab the data with the yfinance package and let's begin!

import pandas as pd
import yfinance as yf

tickers = ['CHF=X', 'BTC-USD', 'SPY', 'GLD', 'USO']
start = '2010-01-01'
end = '2021-12-31'
yfObj = yf.Tickers(tickers)
df = yfObj.history(start=start, end=end)
df.drop(['Stock Splits', 'Dividends', 'Volume', 
         'Open', 'High', 'Low'], axis=1, inplace=True)
df.columns = df.columns.swaplevel()

We're just interested in the closing prices here, so we can drop all of the other columns and run swaplevel() on the columns so we can index by the ticker.

Next, we need to run each time series through our hurst() function to get the exponent and the related values.

vals = {c[0]: hurst(df[c].dropna().values) for c in df.columns}

Once that's complete, I have a helper function to plot the results.

def plotHurst(m, x, y, series, name):
  fig, ax = plt.subplots(1, 2, figsize=(15, 6))
  ax[0].plot(np.log10(x), m[0] * np.log10(x) + m[1]) 
  ax[0].scatter(np.log10(x), np.log10(y), c=colors[1])
  ax[0].set_title(f"{name} (H = {m[0]:.3f})")
  ax[0].set_xlabel(r"log($\tau$)")
  ax[0].set_ylabel(r"log($\sigma_\tau$)")

  ax[1].plot(series)
  ax[1].set_title(f"{name}")
  ax[1].set_ylabel("Price ($)")
  ax[1].set_xlabel("Date")

  return fig, ax

This makes it much easier to plot our values. If we want to plot all of them, we can loop over our vals dictionary and call plt.show() as shown in the code below.

for k, v in vals.items():
  fig, ax = plotHurst(*v, df[k], k)
  plt.show()

Let's look at each of these in turn and see if we can make some sense out of what the Hurst Exponent is telling us.

USD/CHF Mean Reversion

We'll start with the strongest signal we're seeing from our Hurst indicator, the USD/CHF cross.

Most traders have an intuition that currency pairs are mean reverting in general, and it is particularly pronounced for pairs between strong currencies, such as the US dollar and Swiss Franc.

This is exactly what we see here.

What is interesting to note though, is that shorter time frames deviate a bit from the longer time frames we see here. The term structure of the realized volatility (i.e. where the blue dots are) is below the red line on the shortest time horizons.

Just out of curiosity, I plotted the Hurst exponent for a variety of max lag values to see if this series ever becomes a trending series.

It doesn't.

It does get more mean reverting the longer it goes indicating that we have a fairly stable long-term relationship between the USD and CHF.

Keep in mind that this is daily data, so it may provide some guidance as the time frame you should be trading. Seeing as it becomes more mean reverting around 60+ days, perhaps mean reversion strategies that run on that time scale (or longer) would be most efficacious.

Bitcoin is a Momentum Play

Like the Franc/Dollar trade, Bitcoin's boom-bust momentum shouldn't come as any major surprise.

The Hurst exponent gives a fairly clear signal that yes, this is a momentum trade, and the realized volatility term structure shows surprisingly few deviations.

What is interesting though, is the year 2021.

After Bitcoin's huge 2020 bull market, it spent 2021 in a random/mean reversion phase. Note that this is not that much data to go off of, but it is interesting to wonder if Bitcoin has entered a regime change. After the big, 2017 bull market, it entered a similar regime from 2018-2019 where it was mean reverting for two years until it took off in 2020.

So far, 2022 has shown a similar pattern. Could it persist for the rest of the year?

Mean Reversion for the S&P 500

This one came as the biggest surprise to me. The Hurst exponent shows that the SPY is a mean reverting time series.

Looking at it's price chart on the right, it looks like it has a strong, upward trend.

Our algorithm reveals what our eyes don't see.

We can run the same analysis for the S&P 500 as we did for the USD/CHF exchange rate to see if there are any inflection points or changes in the Hurst exponent over different lags.

When we do that, we get the plot below.

In this plot, H changes significantly with the lag. In the short to mid-term, the series is mean reverting, but in the longer term horizon it becomes strongly trending. In the very long term (beyond 400 days) it goes back to be strongly mean reverting.

Let's see if this holds over different time horizons.

We're going to go back and get the SPY from its inception in 1993 through 2021. That will yield 7,284 trading days which we can run this over. Additionally, we'll divide it into four time periods and run the same analysis over each of those four time periods to see if the SPY's behavior changed over smaller intervals.

This yields the plot below:

This shows that the S&P 500 goes through periods of more trending and more mean reversion at different lags. For example, compare the 2000-2007 period. Here we go from dot com boom and bust to the peak of the global financial crisis and nearly all lags are trending. Since then, trending persists to a lesser degree in the ~200-300 range, but mean reversion is more prevalent elsewhere.

An interesting thing to note is that the series seems to be trending in the 200-300 day range in every time frame we look at. This seems to indicate that the S&P 500 exhibits strong trending behavior over those time frames that may be exploited via trend following models, whereas mean reversion models would be more valuable at shorter time horizons.

This observation of trends in the 200-300 day range seems to match with famed Turtle trader Jerry Parker's anecdotal observations that modern trend following works best at these 200-day+ time horizons.

Trading with the Hurst Exponent

Whether a market tends to trend, mean revert, or is just random is valuable information for a trader. While the Hurst exponent isn't an entry signal in and of itself, it can serve as a filter on top of a system. Given that market regimes can shift over time to favor one approach or the other, overlaying your model with a Hurst filter could help prevent your algorithm from buying a breakout in a mean reverting market or shorting ahead of a pullback when the market is moving to new highs.

Essentially, it may help match your algorithm to the markets it is trading.

Of course, this isn't easy and requires a lot of testing. We'll show you how to do just that in a future article. Additionally, we have a no-code algorithmic trading platform that let's you design and test your trading system with just a few clicks. Set it up in seconds, test your ideas, and run your backtests.

Try our free demo today!