Ten years ago, Andres Cortez, a chauffeur in Los Angeles, might have been part of the hordes of people dabbling in day trading or haunting the online stock forums. He might have been bragging to his friends about the money he made in tech stocks, or learning how to margin trade at a night school.

Instead, he keeps his distance from stocks.

As he stands by his car and waits for a passenger downtown, Cortez says he has a little money he's put aside and is keeping it in a savings account, where it earns virtually nothing.

"I read some books about the stock market and stuff, and I don't feel like I understand it enough to really get involved in it," he says. He also saw the documentary Inside Job, about the 2008 financial crisis. "That totally scared me away from the stock market."

Four years after the crisis erupted, the stock market is on the upswing. The Dow Jones industrial average hit a four-year high this week. With interest rates so low, stocks have become the least-worst place for many institutional investors to park their money.

But a lot of small investors are sitting the rally out.

Forty-six percent of U.S. households now have money in stocks or stock mutual funds, down from 59 percent in 2001, says the Investment Company Institute. This year alone, some $70 billion has been pulled out of stock funds.

When people have money to spare, they tend to keep it in savings and checking accounts, though they earn much less money there, says David Santschi, executive vice president of operations at TrimTabs Investment Research, which tracks money flows.

"[The stock market] is not where the real money has been flowing this year," he says. "The real money has been flowing into mattresses, so to speak."

It's not hard to figure out why so many small investors have fled stocks.

"A lot of people are scared and tired," says former Treasury Department official Neel Kashkari, now head of global equities at the bond giant PIMCO. "If you think about the financial crisis, which really started in '07 and exploded in '08, and now the lingering effects ... this has now been going on a long time," Kashkari says. "People are, I think, getting tired of it and are scared."

Kashkari says people just don't want the downside, and say they've worked too hard for their savings. So instead they decided to sit on the sidelines until this passes.

"People got a beating during the [recession], and many small investors don't have a lot of nest eggs to lose," says Sung Won Sohn, professor of economics at California State University Channel Islands. "Many of them lost their retirement funds and 401(k)s."

That bad experience left a very bad taste in their mouths, Sohn says, and they're not ready to jump into the stock market, even if they have the money — which they don't.

Investors have been further soured by incidents like the 2010 "flash crash" and the recent Knight Capital fiasco, when stock prices gyrated dramatically in the space of a few minutes, making the market seem volatile and even rigged against small investors.

This loss of faith in stocks represents a profound psychological shift on the part of investors. Fifteen years ago, it was conventional wisdom that stocks represented the best place to put your money, because they tended to enjoy the highest, most consistent returns over time, a view popularized by Wharton professor Jeremy Siegel, among others. As a result, many millions of Americans put their retirement money into stock mutual funds.

Stocks may still be the best bet for many investors, but it's hard to get that message across to people who have spent years watching their 401(k)s lose ground. Stocks fell so far during the 2000s that someone who bought shares in 2001 may still be behind, Sohn says.

"People can see that, and they've been told stories about the stock market, how wonderful it had been and will be, but that is no longer true," he says.

But Sohn also says investors might be overreacting, going from having too much faith in the markets to having too little. There are still plenty of strong companies that make big profits and pay regular dividends, and they will be well-positioned to grow if and when the economy finally recovers.

Copyright 2016 NPR. To see more, visit http://www.npr.org/.