U.S. markets closed slightly lower Friday afternoon as investors assessed the latest White House shakeup, while shares of sporting goods retailers and Deere weighed on the market following disappointing results.

The White House announced President Donald Trump fired chief strategist Steve Bannon on Friday. Bannon is known as an economic nationalist and an advocate of "America First" policies. However, critics have accused him of harboring anti-Semitic and white nationalist sentiments.

Stocks began the day lower but reversed course as reports of Bannon's departure began to circulate.

"There are still question marks on what this means and what other shoes are to drop, like other personnel of note," said Mark Luschini, chief investment strategist at Janney Montgomery Scott in Philadelphia.

"This is noteworthy, but it's not necessarily economic and therefore the market is viewing it as a positive but only marginally positive, as it doesn't put money in anybody's pocket in the way tax reform would."

U.S. stocks closed lower after another volatile session on Friday as the latest White House shake-up kept investors jittery about the outlook for the Trump agenda.

The Dow Jones Industrial Average fell 76.22 points, or 0.35 per cent, to 21,674.51, the S&P 500 lost 4.46 points, or 0.18 per cent, to 2,425.55 and the Nasdaq Composite dropped 5.39 points, or 0.09 per cent, to 6,216.53. 

All three major indexes are on track to post their worst monthly performance since October, a month before Trump's election as president.

The S&P 500 had posted its biggest percentage decline in three month on Thursday, hit by increased worries about the Trump administration's ability to push through its economic agenda.

Canada's benchmark stock index fell on Friday, led downward by losses in heavyweight bank shares and gold miners, with concerns about the impact of a stronger Canadian dollar and inflation creeping higher weighing on sentiment.

The Toronto Stock Exchange's S&P/TSX composite index ended down 81.31 points, or 0.54 per cent, at 14,952.33. Healthcare was the only group out of the index's 10 key industry sectors that finished higher. 

There was concern about the possible departure of National Economic Council Director Gary Cohn on Thursday, and Trump disbanded two business councils on Wednesday.

Trump has alienated some corporate leaders and U.S. allies with his comments since violence in Charlottesville, Virginia, in the aftermath of a white nationalist protest against the removal of a Confederate statue.

Trump's campaign promises of tax cuts and higher infrastructure spending had helped the market rally, with the S&P climbing about 14 percent since the election. However, investors are getting increasingly worried about Trump's ability to implement his pro-growth agenda.

Nike's (NKE.N) 4.6-per cent slide weighed the most on the S&P and the Dow, following dismal results from sporting goods retailers Foot Locker and Hibbett.

Deere's (DE.N) 5.6-per cent fall was the biggest drag on the industrial sector after the farm equipment maker reported a second straight quarter of lower-than-expected sales.

Traders noted that volume has been light this week with many participants taking vacation before summer's end. "These last few weeks of August; there are lot of people on vacation, so markets are not nearly as liquid as they usually are," said Michael Matousek, head trader at U.S. Global Investors Inc in San Antonio.

Advancing issues outnumbered declining ones on the NYSE by a 1.25-to-1 ratio; on Nasdaq, a 1.16-to-1 ratio favored advancers.