Every year the research firm Dalbar publishes a study that compares investor returns to stock market returns. Almost every year the study shows that investors do worse than the market. The reason seems pretty clear: poor timing.

The latest study looks at the 20-year period from 1992 to 2012:

• Average yearly stock market return = 8.21%
• Average yearly investor portfolio return = 4.25%

If you invested $10,000 in the stock market 20 years ago, and you didn’t touch it for the entire period, it would be worth $46,610 today. But if you tried to “time” the market by buying and selling during those 20 years, your $10,000 investment would only be worth $22,990. Poor timing is very costly.

Why is the average investor so bad at timing? And why do investors even try to time the market in the first place? Psychologists say it’s simply part of human nature. Despite knowing better, we give into the emotions of fear and hope. When the market is going down, we often give in to fear.

But we don’t sell at the beginning of a market decline, we sell when it’s closer to the end. We typically wait until the market has gone down so much that we can’t stand to take any more pain.

And then we compound the problem when the market recovers. We are often late in buying back in. The end result is a classic pattern of “selling low and buying high,” just the opposite of what we should be doing.

So what can you do to change this pattern of behavior?

Like any destructive behavior the first step to fixing it is to admit that there is a problem. Start by taking a full and honest look at your personal investing track record. Don’t rely on memory; look at your account statements.

• Did you sell some of your stock holdings during the market meltdown of 2008-2009?
• Did you get back into the market in 2009?
• Did you sell in 2000-2002? When did you get back in?

Here are some other suggestions.

Have a written plan. Bear markets will always be with us, but when you have a plan you will be prepared when they arrive. If you want to avoid big market declines, start by defining exactly what you mean by “big.” Declines of 10% in the market are common, and it’s nearly impossible to avoid them. But you could create a rule that says “if the market declines by more than 10%, I will sell half of my stock holdings.” This will give structure and discipline to your sell decisions, and it will help take emotions out of the process.

Make a checklist for selling decisions. This list could include things like your 10% rule, your reasons for wanting to sell, and your criteria for getting back in when the coast is clear. It can also include a requirement that you discuss your intention to sell with a trusted friend or adviser before you pull the trigger. The checklist helps to reduce emotion and increase rationality.

Don’t participate in market timing at all. For some investors, the pain of “riding out” a big drop in the market is too much to bear. But for many, especially those who have 10 or more years until they need the money, doing nothing is the best solution to poor timing. Would you rather earn 8% per year, or 4%? If you have the emotional strength to ride out the storms, it could be as simple as that.

In fact, I would say that they’re very bullish. So much so that they showcased their enthusiasm in a “Special Report” inside the current issue. Here are some of the highlights from the special report

“Dow 15,000 may just be the beginning. 17,000 is within range by the end of the year.” (Quote from Jeremy Siegel, Wharton finance professor)

“The odds still favor the bulls.”

“Boost the Dow at 12.2% a year and you get a target of 18,681 over the next two years.”

“The Dow has a 50-50 chance of  vaulting above 18,500 by this time next year.”

What the Smart Money (investment managers at the largest institutions) Is Buying – and Selling

You may scoff at the rampant bullishness on display here, but “at the very least, it seems to have shrunk the hordes waiting for a springtime correction. The latest weekly poll by the American Association of Individual Investors showed bullish sentiment rising for a fourth straight week. Traders are reducing their bearish bets: While the average stock in the broad market is up 6.6% since mid-April, the 150 most-heavily shorted stocks have rallied 9.1%.”  It’s getting hot out there for the bears.

Nothing seems to rattle this market. The old standbys like European debt woes, North Korean missile threats, and continuing weakness in China’s economy – they seem to have taken a back seat to the happy talk about new records being set here at home.

And it looks like investors have started shifting out of “defensive” sectors like consumer staples and utilities, and into “cyclical” sectors like technology and banking. The relentless pursuit of high dividends to offset low rates on treasury bonds could be starting to run out of steam.

Goldman Sachs weighs in on the bullish side

Goldman Sachs’ portfolio strategists, David Kostin and Stuart Kaiser, are telling clients to focus on 25 global cyclical stocks that should outperform the market over the next 12 to 18 months. Some names on the list include Johnson Controls (JCI), Borg Warner (BWA), Marathon Oil (MRO), Analog Devices (ADI), Cisco (CSCO), Qualcomm (QCOM), and Praxair (PX).

The message of all this bullishness seems to be “just ignore the old saying that ‘bulls make money, bears make money, but pigs get slaughtered.’ Go ahead… be a pig.”

Other companies featured in this issue

Western Union should benefit from more customers as a result of lower prices and an improving economy.

Bullish research reports on Ingersoll-Rand (IR), Zillow (Z), Disney (DIS), Cognizant Tech (CTSH), Groupo Televisa (TV), Aetna (AET), and Time Warner (TWX).

A negative review of Uni-Pixel (UNXL), a company that makes touch screens for tablets and smart phones.  It seems the stock has gotten a little ahead of itself as investors anticipate great things for the company, whose products are not yet commercial.  One look at the chart tells the story.  It kind of takes you back to the tech bubble days of 1999-2000, doesn’t it?

 Special Report: ETF Roundtable

Actively managed ETFs are trying to gain traction, but so far they have not succeeded. What are they, and how are they different from “regular” ETFs?  Most ETFs are passively managed, which means that there is no high-priced staff of analysts and portfolio managers picking stocks that they think will beat the market. These ETFs are managed by computers, and they are designed to replicate the returns of an index like the S&P 500.  They are very inexpensive to run, so the investor benefits from a cost advantage over traditional mutual funds.  However, the mutual fund industry has been trying for years to break into the ETF market by claiming that a talented manager can beat the index. It hasn’t worked. Investors are savvy enough to understand that it defeats the purpose of having a low cost alternative to mutual funds.

But they continue to try, because the money in ETFs is staggering.  One interesting comment from the article: “I look at actively managed ETFs not as an investment issue, but a distribution issue.”  Translation?  It’s all about sales, not performance.

To check out this week’s issue, follow this link.

His research on behavioral finance has been supported by the National Science Foundation, CFA Institute, and Investment Management Consultants Association (IMCA) and has been published in the Journal of Finance, Financial Analysts Journal, Journal of Portfolio Management, and many other publications.

Statman consults with many investment companies and presents his work to academics and professionals in the U.S. and abroad.

The following excerpts were taken from an interview I saw in Annuity Digest. (Click here to read the full interview.)

People in the investment field are generally aware of behavioral finance, but I worry that they misperceive it. I worry that we have moved from the cardboard image of investors as rational to the cardboard image of investors as irrational and lost the true image of normal investors in the process. Normal investors are investors like you and me, often normal-smart but sometimes normal-stupid. I also worry that we have lost the connection between investments and the life of investors beyond investments. I wanted to understand what normal investors really want. And I wanted to describe normal investors as they really are. Because investors who fail to understand themselves cannot help themselves, and advisors who fail to understand normal investors cannot help them.

Standard finance describes investors as computer-like rational people, doing everything right. Much of behavioral finance describes investors as bumbling irrational people doing everything wrong. It is time to describe investors as normal, often right and often wrong.
I also worry that investors see behavioral finance as a tool for beating the market by taking advantage of the errors of others, rather than protect themselves from themselves.

Normal investors want more from investments than profits equal to risks. We want to nurture hope for riches and banish fear of poverty. We want to win, be #1, and beat the market. We want to feel pride when our investments bring gains and avoid regret when they inflict losses. We want the status conveyed by hedge funds and the virtue conveyed by socially responsible funds. We want financial markets to be fair but we search for an edge that would let us win. We want to leave a legacy for our children when we are gone. And we want to leave nothing for the tax man.

The benefits of investing are utilitarian, expressive, and emotional. Utilitarian benefits are the answer to the question: What does it do for me and my pocketbook? The utilitarian benefits of watches include time telling, the utilitarian benefits of restaurants include nutritious calories, and the utilitarian benefits of investments are mostly wealth, enhanced by high investment returns.

Expressive benefits convey to us and to others our values, tastes, and status. They answer the question: What does it say about me to others and to me? Private banking expresses status and esteem. A stock picker says, “I am smart, able to pick winning stocks.” An options trader says, “I’m sophisticated, willing to take risk and knowing how to control it.”

Emotional benefits are the answer to the question: How does it make me feel? The best tables at prestigious restaurants make us feel proud, insurance policies make us feel safe, lottery tickets and speculative stocks give us hope, and stock trading is exciting.

We face tradeoffs everywhere. A man can take his date to a local Italian restaurant where he would pay $30 for 1,000 utilitarian calories for each of them. Or he can take her to a fancy French restaurant where he would pay $300 for the same 1,000 utilitarian calories plus expressive and emotional benefits. The man gains expressive benefits as he expresses to his date his riches and generosity. He gains the emotional benefits of pride in himself and the hope that this night’s date would lead to more. Sometimes the choice of the expensive restaurant is best. But we must know the tradeoffs between the three kinds of benefits. We can’t have it all.

People who have pensions, annuities or other guaranteed income are happier in retirement and show fewer symptoms of depression than people with equal income from sources which are not as assured.

Most of us find different channels from which to derive expressive and emotional benefits. Think, for example, about social status which is an expressive and emotional benefit. People choose their arenas for status competitions. Hedge fund managers compete with hedge fund managers and academics compete with academics. People with little status in their day jobs might find status on the golf course during the weekend. Yet other people insist of competing for status in the investments arena.

There is hardly a utilitarian case to be made for individual investors buying active mutual funds and playing other beat-the-market games. The same is true for many institutional investors. So why do investors persist in playing the beat-the-market game? One part of the answer is in cognitive errors and misleading emotions. Think about framing. Investors often frame the beat-the-market game as playing tennis against the practice wall. But the game is real tennis, with possibly world-class opponents on the other side of the net. Would you play tennis against Venus Williams if $100,000 would be paid by the loser to the winner?

The other part of the answer is in the expressive and emotional benefits of the beat-the-market game. Index investing is “mediocre.” We hate to be mediocre. We want to win, even if we are likely to fall below mediocre when we play the game. Playing the game gives us hope, like buying a lottery ticket, even if the game is stacked against us. And we enjoy playing the beat-the-market game as we enjoy playing video games or skiing. We have no trouble admitting that we ski for expressive and emotional benefits even if it costs us utilitarian money. But we have great trouble admitting, even to ourselves, that we play the beat-the-market game for expressive and emotional benefits.

There is a big difference between losing $1,000 in Vegas and losing our retirement savings on foolish investments. This is why I think that many of us would benefit from a dose of paternalism, despite the howls of libertarians.

Think about Social Security—what if Social Security is privatized and some people gamble away their money. Are we really ready to say tough luck, now you’ll live in the gutter? I’m not ready to say that.

I think defined contribution plans such as the 401k are an essential part of our retirement system. The defined benefit system is not sustainable.
What we do need is to strengthen the defined contribution system. For example, some countries have a three-leg system consisting of a state-sponsored program similar to Social Security, mandatory savings, and voluntary savings.

In the United States, the second and third legs are lumped together in a 401k and IRA since the contributions to these plans are voluntary.
What this means is that many people are condemned to live entirely on Social Security, and this is not viable.
Participation in 401k should be mandatory and minimum contribution levels should be set.

Just think about defined benefits in the context of public employees and the type of corruption it invites. Politicians make promises of future pensions that are beyond what taxpayers can pay and then simply pass the problem along to other politicians.

The state of California is a perfect example—the game is corrupt since it is based on unsustainable promises. What we need is a transparent system. A policeman might have a salary of $80,000 per year plus opaque benefits worth another $80,000. Let’s make sure that we all see the $160,000 total and that it comes out of today’s budget. This will invite the proper backlash from taxpayers, forcing politicians to confront unions and reduce total compensation to reasonable levels.

Again, I would advocate a mandatory defined contribution system for all employees in both the public and private sectors.

Mandatory savings makes sense and there is a case to be made for taking away investment discretion now available to individuals.
Government guarantees beyond Social Security do not make sense. We should, as a nation, provide a minimum standard of living for old people, but no more than that. (Think of the Medicare money we, taxpayers, must pay to keep Joe’s 98-year old Grandma on life support because Joe refuses to let us pull the plug yet cannot be compelled to pay for his Grandma’s care.)

Managed withdrawal programs do not eliminate longevity risk. I challenge you, though, to find a middle class person who has run out of money in retirement. People are not stupid—they adjust their spending when there is a downturn in the stock market or when they see that they are spending too much. Moreover, people often have a buffer in the form of home equity they do not count as part of their portfolio but is available if needed.

Some people say that the comparison between advisors and physicians does not hold because physicians know much more than advisors. But I think that our estimates of physicians’ knowledge are biased upward and our estimates of advisors’ knowledge are biased downward. Both professions have fiduciary responsibilities and it is not clear to me that physicians fulfill their responsibilities any better than advisors.

The similarities are many. Good physicians promote health and well-being and good advisors promote wealth and well-being.

For one thing, they get the adviser’s full and focused attention.  Top advisers only handle a small number of relationships because clients at that level demand what one of my former employers called “high-touch” service.  They get pampered with “concierge” services – everything from floor seats at the NBA games to private lunches with industry luminaries like Buffett and Gross.

They expect high quality market advice, winning stock recommendations, frequent contact, and access to private stock deals and hot new issues.  Most investors never see or even hear about these kinds of perks, because most investors deal with advisers who have too many clients.  Every adviser I know is trying to break into the top echelon, and the way to do that is by constantly upgrading their “book” of clients.

Advisers compete for the biggest clients they can find, and when they land a big account, they shed some of their smaller clients to make room for the new relationship. This culling-of-the-herd process creates an environment where small- to medium-sized clients are repeatedly being shuffled from one adviser to the next.  Experienced advisers who are moving up the food chain pass them to less experienced ones who are usually just fighting for survival.

I’m often asked to give free advice.  I do, but I have to be selective about it because my time, like yours, is limited.  In addition to my immediate family, I sometimes offer free advice to prospective clients on this website with the hope that some of them will one day become coaching clients.

So, how much is free investment advice really worth?  It depends on how well you understand where to find the right information for the task at hand.  The advice I offer for free on this site is valuable, but unless it’s used in the context of an overall investment strategy, it’s usefulness is limited.  For example, a prospective client recently asked me to help her choose a bond fund from the list of choices in her 401k plan.  I did the research, and based on criteria such as quality, diversification and fees, I made my recommendations.

The following month she became a client.  After looking over all of her investments, and placing everything in the context of her financial “big picture,” it was apparent that she should not have purchased any of the bond funds on her list.  She already had too much of her money invested in bonds, and what she needed was more exposure to stocks.

This is a case where the advice I gave for free was honest and correct, but it was out of context.  So why I didn’t ask her earlier about the context?  I did ask, and she chose not to disclose her financial information out of a concern for privacy.  That’s fair enough, but it meant that the good advice I gave her actually made her financial situation worse.  It caused her portfolio to be even more unbalanced than it already was. This is something to consider when you search the web for advice about how to invest your money.

How valuable is your time?  Like so many other complex issues in life, managing your investments is time consuming.  In my coaching seminars  I provide a step by step plan on how you could do your own investing work. Given enough time, anyone can read and learn how to invest, and use a combination of online calculators and excel spreadsheets to set up a retirement portfolio. But is that the best use of your time?

Although I know I am capable of doing my own tax return, it is not worth my time. I pay my CPA to know the rules, use them to my advantage, and save me the hassle of doing it myself. I am also much more likely to make costly errors than she is, and doing taxes is not my profession. I pay her fee because it’s worth it to me. I save time, avoid frustration, and undoubtedly pay less to the IRS than I would if I did it myself.

When I was younger I used to do my own home repairs and remodeling projects. I had more time than money. I learned from scratch, and I enjoyed all of it. But when I got older, I had less free time for those things. Now I would much rather hire a competent professional to do things that require more time and effort than I’m willing to put forth.

If you have the time, and you enjoy doing your own research, you can do very well with your investments. Free advice and information is out there, but you have to know what to look for, where to find it, which sources you can trust, and how to use it in the proper context. Each of these steps in the Do-it-yourself process introduces a potential for error or misjudgment. You have to get it all right in order for free advice to have any value.

Finally, here’s a suggestion that works for many of my clients. Instead of thinking of advice as an either/or proposition, consider hiring a professional for some of the tasks on a limited basis. I have a number of clients who use me to evaluate how they’re doing now, make suggestions about how they can improve, and help them through the transition. These limited engagements can last from 3 months to one year, but they do come to an end.  Once the client is up and running, it’s much easier to maintain a well-designed portfolio than it is to build one from scratch on your own.

If you would like to find out more about limited engagement coaching, click here.

Every year Barron’s publishes a list of what they consider the best 500 stocks in the market. Here is the methodology they use to come up with the list.

The Barron’s 500 is an exclusive ranking of the 500 largest publicly traded companies in the U.S. and Canada, as measured by sales in the latest fiscal year. The survey is prepared by HOLT, a unit of Credit Suisse, which grades and ranks all companies on the basis of three equally weighted measures: median three-year return on investment, based on a proprietary cash-flow metric called CFROI; the change in CFROI in the latest fiscal year relative to the three-year median; and sales growth in the latest fiscal year, adjusted for pro forma results (and, in the case of cigarette companies, taxes that are collected and remitted to the government).

HOLT’s CFROI metric strips out the effects of inflation and adjusts for accounting distortions, including the  capitalization of research and development, and off-balance-sheet operating leases.  For financial companies, the firm calculates cash-flow return on tangible equity.  All data are based on the companies’ latest reported fiscal year, which for most is  2012.

Each company is graded in three categories; the top quintile gets an A, the bottom, an F. HOLT then calculates the total grade-point average, or GPA, for each company, with 4.0 the highest. In cases where scores are tied, the ranking is based on one-year CFROI relative to the three-year median. The Barron’s 500 excludes companies restating financial data or operating under bankruptcy protection.

This year, Apple is #1. If this surprises you, you’re not alone. The stock has lost some of its’ stature, and it has come down in price from $705 to $400, but the fundamentals remain strong, according to the screening algorithms used by HOLT.

#3 on the list is Western Digital, and #6 is Qualcomm. The fact that these three companies are ranked so high is an indication of how out-of-favor tech stocks are, given the current preference for steady, high-yielding stocks.

Here is the complete list.

Have you ever bought a stock based on your “gut feeling” that the company was a winner?  You assumed that the stock price was bound to go much higher when other investors started to recognize what you can plainly see.  We often are tempted to buy stocks because we think we see something that others haven’t yet seen.  We think we know something that others don’t know.

Maybe you work in the same industry, so you think you know more than the average person about how to recognize winners.  After all, didn’t Peter Lynch, the superstar manager of Fidelity Magellan Fund, once say that investors should “buy what you know?”

Lynch argued that having a job in the furniture business, for example, meant that you probably know more about furniture stocks than most investors, and therefore you had an “information advantage.”  The problem is that the edge you have cuts both ways.  

Your “insider” knowledge of your industry gives you a better sense of how the industry works, but it can also give you a false sense of confidence in your ability to pick winning stocks in that industry.

Nate Silver, the statistician and author of the best selling book “The Signal And The Noise”, commented recently about the problem of overconfidence.  Silver explained that the reason so many “experts” get things wrong is, in large part, because they are experts.

In other words, they know too much.  And because they spend so much of their time thinking about their field of expertise, they sometimes confuse their personal views with facts. 

“Most things are fundamentally too complicated to easily guess the outcome,” Silver explains. Overconfidence “causes us to discount the risks and plunge ahead, only to find out later how wrong we can really be.”

Does this mean that you should never buy stock in one particular company?  Not necessarily.  But long-term investors should understand that the “gut feeling” they have, about a company that’s part of the industry that they understand well, is really only about 50% knowledge.  The other 50% is probably pure emotion.

Certainty is an emotion, and it’s one that can cloud your judgment and blind you to risk. Anybody who has ever bought a stock at “the bottom,” and then watched in horror as it kept on falling, knows this feeling all too well.

“How could the market be so wrong?” you think to yourself, “Why don’t other investors understand how great this company is, and how ridiculously cheap the stock is?”

The “market” may not know what you personally know. But it also doesn’t know what you mistakenly believe to be true.

The best way to put your “gut feeling” to work for you in a constructive way is to consider it a “leading indicator” of the need to investigate further. Listen to your instincts about a stock, but always resist the urge to act on instinct alone.  Ask yourself how much you could lose if your instincts turn out to be wrong.  Ask yourself how likely it is that you alone are correct about the company, and thousands of other investors are wrong.  

By combining instinct with further research, you can reduce the number of emotionally-driven investment mistakes that you make.  This alone will make you a better investor.

1.  Make sure you have enough money in your emergency fund.  The stock market will go down, but your emergency fund (in the form of cash, money-market funds, Treasury bills, CDs, or other stable assets) will not.  If the next recession is worse than usual, one of the biggest risks you’ll face is a loss of income (layoff, or loss of employment.)  Having a 6 month cash cushion will help you ride it out.

2.  Rebalance your portfolio faithfully, according to your written plan.  And at the first sign of trouble, start rebalancing even more frequently.  Most of my clients are on a quarterly schedule, but you can do this once per year and still be in good shape.  But when a recession is threatening (I send email alerts to all of my subscribers and coaching clients), it’s important to speed up this process.

3.  Have a Plan B in place.  All of my clients have a written plan, including steps to take when the economy gets into serious trouble.  The benefit of having a contingency plan in place is that it keeps you from making emotional trading decisions.  For example, some clients set up tripwires based on the likelihood of a recession.  Others use percentage declines in the value of their portfolio.  When you have tripwires in place, and you update them regularly, you will know what to do and when to do it.

4.  Add an extra asset class to your portfolio mix.  Many investors use 3 asset classes – stocks, bonds, and cash.  For long-term investors this is adequate.  But adding a 4th asset class will improve your diversification and act as a buffer against market volatility.  Consider real estate investment trusts (REITs), Master Limited Partnerships (MLPs), Absolute Return Funds, Managed Futures Funds, Commodity Funds, Option Buy-Write Funds, etc.

5.  Don’t panic.  Without question, the costliest mistake people make is panic selling.  Even people who have a written plan can fall into this trap.  When the market is falling apart, it’s very hard to resist the fight-or-flight instinct.  Self-preservation is a powerful emotion to overcome, and that’s exactly what contingency plans are there for.

If you would like information about creating or updating your written plan, send me a message at info@zeninvestor.org or fill out the Customer Contact form.  There is no charge for an initial consultation about planning, and you will not be required to sit through a sales pitch from me.  I promise to answer your questions to the best of my ability, and it will be up to you to decide whether it’s worth pursuing from that point.