Yes, some retail investors can and do beat the market. Doing it consistently is hard.
To beat the market over time takes a combination of skill, discipline, and luck.
In this article I'm going to offer a few tips about what it takes to be a consistently successful stock picker. Why listen to me?
First, I've been in the game for more than 30 years as a professional, and to borrow from the insurance commercial, "I know a thing or two because I've seen a thing or two." I have a knack for picking stocks. I acquired this knack by watching my dad pore over the stock tables every morning when I was a kid. He was very good at picking winners, and I learned a lot from him.
Second, I have an established track record. (You can see it here.) I'm a professional, and I came up through the ranks, starting as a junior trader for a regional brokerage firm in Milwaukee. Most junior traders washed out within a few months.
After a year on the trading desk I was promoted to Senior Trader, which meant that I was able to use firm capital to take trading positions. It wasn't a lot of money - maybe $300,000 in today's dollars - but it was a start.
Shortly thereafter, my firm was purchased by Kemper Securities - the brokerage division of the giant insurer Kemper Corp. They wanted to diversify their boring 5%-6% bond portfolios and add a little equity juice to the mix.
Kemper assembled a team of seven people to establish a proprietary trading desk that was funded with firm capital. The brains of the operation was Owen Hill, a certified genius and one of the most respected and admired men in the firm. The head of research was Ron Bukowski, another genius who had a talent for finding the next great companies.
My role was co-manager of the portfolio and Head Trader. I had two junior traders who reported to me, and the other two team members provided support for compliance and accounting.
Skill
Our team certainly had skill, but skill is hard to define when it comes to picking winning stocks. Aside from the obvious - a long-term winning track record - skill involves a few key characteristics.
One of these is a natural skepticism about the stories that are touted by Wall Street analysts and investment bankers. Hype is everywhere and a skilled investor knows how to cut through the hype and find the real story about a company and its prospects.
Discipline
We had a very disciplined process for picking stocks. All great stock-pickers share the trait of discipline. Without it, you're just operating on instinct, and instinct can fail you at any moment.
Regardless of what your goals and intentions are, you need to have a plan. And once you have a plan, you need to stick to it, no matter what the circumstances. Establish and follow a discipline, which really means just doing what you said you are going to do.
Give every investment in your portfolio a buy price and sell price along with your reasons to buy, hold, or sell at that price. This gives you specific criteria to act and provides your portfolio with purpose and direction.
[Investopedia]
Luck
And we were lucky. Luck is easy to define, but it plays a meaningful role in successful stock-picking. In a world where companies like Enron, WorldCom, and Bernie Madoff's managed funds were eventually revealed to be scams, it takes more than a little luck to avoid these types of investment landmines.
The Odds of an individual (retail) investor beating the market.
There are roughly 100 million retail investors in the U.S. As a group, they underperform the professionals who manage very large pools of money. But they do have an advantage - nimbleness. But being nimble often leads to over-trading, which is a return killer.
Large mutual funds and ETFs are constrained by trading liquidity, but retail investors aren't. They are free to buy small, illiquid stocks. Additionally, large managers underperform the market, which leaves opportunities for retail investors to outperform.
According to Terrance Odean, a finance professor at the University of California, Berkley's Haas School of Business, "Many of the mistakes investors make come from a lack of any understanding of the innate disadvantages they face."
The average investor may not have a very good chance of beating the market. Individual investors may be able to achieve better risk-adjusted returns by focusing on managing risk (losing less).
By beating the market we're talking about everyday working Americans who invest to try to get higher returns than the S&P 500.
"We all buy securities because we think we know someone or something others don't. I don't think anyone can consistently outperform the S&P 500 without assuming greater than market risk," according to David E. Y. Sarna, author of "History Of Greed." Investors are set-up to fail from the very beginning.
All the evidence supports the disappointing fact that regular investors as a whole underperform the market. As long as they try to 'beat the market' they actually underperform.
But there's hope. Individual investors are learning that stocks aren't as scary as everyone suggests and there is valuable information available to everyone if they know where to find it and how to apply it.
The advent of ETFs and index investing allow people to mimic the market, instead of trying to beat it, which is a better, less expensive perspective to have,
What can an investor do to increase their chances of beating the market?
Ivkovich, Sialm and Weisbenner (2005) find that individuals with relatively concentrated portfolios outperform those that are more diversified, and Kaniel Saar and Titman (2005) find evidence that stocks that are heavily purchased by individual investors in one month exhibit positive excess returns in the 6 months following. They also report evidence suggesting that individual investors trade unprofitably in response to cash flow news and earnings announcements.
These findings raise the question of whether some individual investors have superior skill, and are able to profit thereby. Of course, given the low diversification of many of the accounts, one expects substantial cross-sectional variation in account performance by chance even if there are no differences in skill.
The debate of whether an individual investor can beat the market is as old as the stock market itself. Those who have found fortune investing will often preach that they possess superior analytical skills which allowed them to predict the market. Those investors who suffer losses will tell a much different tale.