Your Professors are Wrong: You Can Beat the Market

If you’ve had any academic training in economics, you have likely been told that nobody can “beat the market” because markets are efficient.

Right out of the gate your hopes and dreams of becoming the next Jim Simons or Warren Buffett are dashed by the Efficient Market Hypothesis (EMH). In fact, according to the EMH, their performance is the result of blind luck; if you have enough monkeys banging on typewriters, you will eventually produce some Shakespeare.

The EMH (or at least the strong version) states that all information has been priced into stocks and securities. In other words, there’s no way to consistently beat the broad market averages. Your best bet then is to passively invest in a low-cost index fund and just let the market do its thing.

One of the theory’s strongest proponents, Burton Malkiel, writes:

EMH and Random Walks

Proponents of EMH argue that the market moves in a “random walk” that can be neatly described by equations borrowed from physics, i.e. Brownian Motion. This follows from the EMH. If investors are pricing in all information, both public and private into their decisions and forecasting rationally, then only new information — which is random and unforecastable in nature — is the only thing that can move prices. Thus, prices move randomly.

This assumption underlies most financial modeling and risk measurements, despite some spectacular failures.

The academic edifice of EMH seems daunting — Nobel Prizes have been bestowed on its developers! Despite this, the EMH remains a deeply flawed theory; thankfully so for intrepid investors seeking to beat the market.

Rational Expectations — Irrational Investors

A core problem of EMH is that it relies on rational expectations theory, a theory that states people are free from biases in their decision making and are utility optimizers.

Unfortunately for the theory, investors are far from rational actors — take a minute or two to watch TikTok investors or read some posts on just about any investing message board. To call some of these discussions “rational” stretches the meaning of the word beyond recognition.

Contra rational expectations, investors — who are just normal people after all — are fraught with cognitive biases. These biases can’t simply be assumed away to make the stock market math more manageable (Nobel Prizes have been awarded for work on these cognitive biases as well).

EMH: Simplifying Investor Decisions

If the EMH is correct, however, the price right now is “correct” in that all information is baked in and properly discounted. If one investor thinks a stock is undervalued (again, according to her cold, rational analysis), then she is going to bid the price up to the point she no longer considers it undervalued. Likewise if one believes a stock is overvalued, he’s going to push it down until it is no longer overvalued by selling or shorting the shares.

There is some plausibility (and truth) to this story, but only partially because it ignores real constraints that investors have.

Take for example, someone who believes firmly that Tesla is massively overvalued — there seems no shortage of such short selling bears. If they could really bid the price down accordingly, they would. However there remains far too much buying pressure on the other side of these trades for even large funds and groups of investors to move the price significantly lower.

To borrow another example from Bob Murphy. Imagine you get a time machine and can jump forward five years and see that the best performing stock over this period increased from $1 today to $500 five years from now. According to our EMH and rational expectations theory, you should value that stock at $500 today (ignoring discounting for simplicity’s sake) and buy every share until it reaches that price. You may leverage your home, max out your credit cards, get friends and family to invest, and throw every spare dollar you can into this stock, but how high could you really move it? If you’re like most people, you simply won’t have enough capital to significantly move the price on your own despite your perfectly rational expectations.

Markets are not Random

Let’s pick on Prof. Malkiel again. In his famous book, he provides an anecdote whereby he proceeds to flip a coin for his class to determine the daily price of a hypothetical security. Heads yielded a slight rise, tails a slight reduction in price. Over time, a chart was developed using this random generation process, when the good professor brought his creation to a chartist. The chartist suggested that this stock should be bought and had a strong, bullish forecast for this random stock. Based on this prognostication, Prof. Malkiel concluded that stocks are indistinguishable from random processes.

It’s a nice story, but Malkiel made a number of critical errors in his conclusion. First, while the chartist was unable to visually differentiate between a random process and actual stocks, it could just as well be that the random pattern Malkiel created was an excellent forgery; repeating this test may have yielded different results. Second, while the human eye may not have been able to distinguish between a randomness and an actual time series of prices, that doesn’t mean a computer can’t. In fact, auto-correlation — the tendency for stocks to trend in a given direction over time — is a well known phenomenon and goes sharply against EMH and the random walk hypothesis. Finally, even if stock prices stand up to strong tests of randomness, it doesn’t mean that additional information (e.g. volume, earnings or fundamental data) couldn’t reveal important, non-random patterns in prices.

In short, Malkiel’s test was too simple and he reached his conclusion far too hastily.

Too Many Outliers to Count

Perhaps one of the most difficult issues to reconcile with the EMH worldview are the long list of outliers. Both market events — typically crashes — and highly successful investors with long track records which should not exist if the EMH were true.

We have the October 1987 crash, a single day loss that the markets have never seen before or since. The blow-up of Long Term Capital Management (LTCM), a hedge fund founded by Nobel Prize winners in a series of events that should have never happened if their theories were correct. My favorite example comes from the Great Financial Crisis where the CEO of Goldman Sachs said they experienced losses from a series of 25-sigma events!

If you don’t know how often a 25-sigma event should occur, we have a handy table available:

The authors, Dowd et al. give some context for these numbers:

It seems safe to say if you experience a few events with a 1/1.3019 x 10¹³⁵ chance of occurring, your model might have some faulty assumptions.

On the other hand, there is a long list of investors who have consistently beat the market year in and year out. Yes, some of this is certainly luck, but the longer your track record, the less likely luck is at play. Luck simply cannot account for everything. If the EMH was true, these people should not exist.

Market Inefficiencies are Available to You!

Thankfully for us, markets do exhibit inefficiencies. These enable savvy investors to outperform the market. Unfortunately, this is easier said than done.

We try to make it as easy as possible for investors by providing robust tools using high-quality data to allow you to develop your own quantitative strategies. You can design your signals, adjust your risk, and test your strategy to see how it performs in a variety of market environments. When you’re happy with the results, just deploy the system and you’re ready to trade it!

Check out our free demo here.