Tuesday, September 30, 2008

Hindsight Bias and Regret in Trading

"I should have caught the move"

"I knew that was going to happen"

"If only I had followed my signal"

When markets become unusually volatile, they make unusually large moves. To the short-term trader or the active portfolio manager, such moves look like phenomenal opportunity. This creates a kind of dissonance when their results do not reflect such opportunity. This dissonance is often expressed as regret: the word "should" becomes a prominent part of traders' thinking.

Underneath this regret is what behavioral finance researchers call "hindsight bias": the exaggerated sense of predictability in retrospect. Very often, traders will have reasons in mind why the market might rise and they can identify reasons why it might fall. The evidence from research, charts, fundamentals, and indicators often paints a mixed picture. In retrospect, however, traders will look back on market outcomes and selectively pick out the evidence that would have predicted the market's movements. They minimize the ambiguity that occurred at the time and convince themselves that they knew all along what the market was going to do.

It's easy to see how hindsight bias and regret go hand in hand. If you convince yourself that you saw the market move in advance and you see that you didn't participate in the move, the dissonance between what your profitability should be and what it is leaves plenty of room for self-recrimination. Out of this regret, traders often feel pressure to make up for the "missed opportunity", leading to overtrading.

A psychodynamic psychologist would view hindsight bias as a kind of defense: it protects traders from the anxieties of ambiguity and unpredictability and reinforces an illusion of control. A number of behavioral finance investigations have shown traders charts composed of random price movements; invariably traders find meaningful patterns in the randomness. For them, the anxiety may not be the market going up or down; the anxiety is not knowing what the market will do.

It takes a strong psychological constitution to tolerate such ambiguity and uncertainty. And yet, it is precisely the embrace of the market's uncertainties that allows us to be alert to risk and implement proper risk management. Hindsight bias appears to be a natural response to an updating of information regarding events; it's part of how we make sense of our world. As Richard Peterson notes in his book "Inside the Investor's Brain", research from Paul Slovic finds that the best antidote to hindsight bias is a hard, purposeful look at "counterfactuals". Once a market event occurs, considering the array of alternative outcomes and their implications helps moderate hindsight bias and associated regret.

Given the limits of what we know and what is ultimately unknowable, not all movement is opportunity. The key to trading success is finding the patience to capitalize on those things you do know and the wisdom to accept what is uncertain.
.

7 comments:

Newton Linchen said...

Bret,

You just nailed it!

Congratulations and THANK YOU for this excelent post!

Actually, in my own case, it took many years to finally get rid of regret and this hindsight bias.

I thinks it's about being humble. If you're NOT humble, (and the traders, by definition, are usually not), then the MARKET will make you humble.

After many years of suffering, I realized that you can't be RIGHT at the markets. You can't PREDICT without FAILURE what's going to happen. You can just try to SEE what's going on and, if you're skilled, get along with the flow.

My great mistake at the markets was always the NEED to be "right". I costed me much money and suffering.

Finally, then, I realized that the MARKET was not my father, in front of whom I "had" to be "right".

The MARKET is wonderful because it's impersonal. It's almost like a force of NATURE, it doesn't know you, it doesn't care if you're profitable or deep in the red.

It just exists.

When you finally come to this understanding, it's pretty liberating. You finally feel free and can place your trades like free movements of yourself.

You stop arguing with the markets. You stop punishing yourself for "not being always right".

That's my view.

Newton Linchen, Brazilian Trader

The Stock Speculator said...

Brett,

Thanks for putting a name to it. When I started to trade, I was the guy always beating myself up about missing the move.

Once I started to keep a log and track my thoughts and compared them to my actions, I realized that it was all in my head.

Yes I missed good trades and could rightfully say- I saw that, but trade rules kept me away. I also missed horrendous trades that would have blown right up.

The brain is a funny thing. I never ruminate on how I just missed a bad trade by sticking to my trading rules. It only focuses me on the profitable trades I missed and tells me to ignore the rules.

I guess we all have a self destructive tendency or three that we need to be aware of.

Great post as usual.

TSSS

Kevin said...

Excellent post Brett. You have a wonderful way of putting down in words the actions and thoughts that traders make on a daily basis.

Kevin

market folly said...

great writeup!

although your post is slightly different, it made me think of this quote: "the opportunities that are clear in retrospect are rarely visible in prospect"

Don Miller said...

Great post Brett.

I've also found that using the pain of regret (and its lasting memory) to force action the next time a similar situation occurs to avoid such pain can he helpful ... often leading to a daily "no regrets" motto.

Don Miller

Krasimir said...

Brett,

Great post! In addition to what you wrote I would say that hindsight bias is observable in backtesting as well, especially if trading rules have some subjectivity. It is easier to spot patterns in a historical data, and it is a different thing when you have to recognize them in real time. That’s why sometimes backtesting results are superior to the real time one, given the fact that market behavior and trading skills are being constant during that observation.

Wojtek said...

Mark Douglas states that in every single moment you know everything. As your knowledge and market ability develops, you may see opportunities NOW, but that does not mean you could have seen them before.

If you could have, you would have - it's that simple.

It's easy to be a market wizzard in the middle of the chart. A four-year-old could do it as well.

Good trading

W