Markets just can’t seem to get on the same page as chief executives of Canada’s top banks when it comes to the cooling of the nation’s hottest housing markets. The TSX financials remain essentially flat on a year-to-date basis in part due to concerns the big five banks will take a hit due to a drop off in activity and prices in Vancouver and Toronto.

Canada’s banks say the lenders will not only be able to withstand the cooling, but see the slowing as a positive sign for the long-term stability of the overall economy.

“On Canadian housing, we’ve seen a healthy moderation in housing activity in Toronto and Vancouver, supported by policy actions by provincial governments and our regulator to create a healthier, and more balanced market,” RBC Chief Executive Officer Dave McKay told investors on the company’s third-quarter conference call. “We continue to prudently grow our mortgage portfolio with strong FICO scores at origination, and are seeing low delinquencies.”

FICO measures the creditworthiness of consumers and RBC (RY.TO) says nearly half of those with a mortgage with the bank have FICO scores at the high end of the range. The number of borrowers who are more than three months behind on their payments in Toronto and Vancouver lag the national average, the bank says.

In spite of McKay’s reassurances, investors don’t seem confident enough to buy into the stock. Shares of RBC are up a paltry 2.8 per cent over the course of 2017, in line with the tepid performance of its counterparts as Toronto housing prices remain 20 per cent off the peak hit earlier this year.

At least one Bay Street money manager thinks the fears of a housing crash are being unduly priced into the nation’s major lenders. In an interview on BNN, Brian Madden, Senior Vice President & Portfolio Manager at Goodreid Investment Counsel, said the issue is the single largest impacting investor sentiment surrounding the big banks.

“This issue of mortgage risk and credit risk is the biggest white elephant in the country bar none: it’s gigantic,” Madden said. “Although housing is cooling off in Toronto and had a brief period of cooling off in Vancouver late last year, Canadian housing markets, Canadian mortgage markets are just not the same sort of animal as their U.S. counterparts. We just don’t see the spectre of elevated credit losses.”

While markets continue to punish the banks for their residential mortgage exposure, the smallest of Canada’s big five banks has been doubling down on Canadian homebuyers. Average mortgage balances at CIBC surged 13 per cent year-over-year in the third quarter, and the total amount of Canadian mortgages on the books hit $197 billion. CIBC (CM.TO) says it expects higher interest rates and new regulatory measures to moderate that pace in the coming quarters, and says it’s comfortable with its exposure to the sector.

CIBC said a widespread loss of employment would be a much bigger blow to the division than any swoon in prices. On the company’s third quarter earnings conference call, Chief Risk Officer Laura Dottori-Attanasio said the underlying economic strength in the domestic economy gives the bank confidence it’s made the right bet.

“When you do look at our loan to values, I guess what I would say is, we do have adequate buffer to sustain a drop in housing prices,” she said. “We need our borrowers to continue to have their jobs to service their loans. But in the event we find ourselves taking on assets, we do have and continue to have a good buffer as it relates to that loan to value.”

The sentiment appeared to fall on deaf ears in Thursday’s trade, with shares of CIBC falling nearly 3 per cent before posting a modest rebound Friday.

That overall weakness has one money manager recommending investors consider some bottom fishing on the banks. In an interview on BNN, Douglas Kee, Chief Investment Officer at Leon Frazer & Associates said he’s allocated 20 per cent of his portfolio to Canadian financials, as he expects mortgage growth to merely slow, not shift into reverse.

“Their mortgage portfolio growth is going to slow: the banks have said that,” he said. “It’s just a slowdown in growth, it’s not that things are going bad. The bubble isn’t bursting.”