Muhyiddin had said at the weekend the ruling coalition could lose the next election if Najib and others did not better manage issues stemming from the scandal at 1Malaysia Development Berhad, which has debts of more than $11bn.
Muhyiddin has been replaced by Home Minister Zahid Hamidi, a right-wing politician well-liked by members of the United Malays National Organisation party, which has led Malaysian coalition governments since independence in 1957, Al Jazeera reported.
"The decision to replace Tan Sri Muhyiddin Yassin was very difficult, but leadership is about doing what you think is right," Najib said in a speech on Tuesday.
"Nevertheless, I thank Tan Sri Muhyiddin for all his work and dedication to the government and country, as I do all those ministers who are no longer in the cabinet."
Bernama news agency reported on Tuesday that Najib also sacked Attorney General Abdul Gani Patail, who was involved in investigations into corruption allegations related to the Najib-linked company.
The prime minister, who launched 1MDB in 2009, has vehemently denied any wrongdoing.
Najib told his deputy and other officials on Monday to stop commenting on the scandal at the investment fund, saying arguments among leaders would erode support for the government.
Corruption allegations
Muhyiddin had repeatedly backed mounting public calls for Najib to answer allegations that hundreds of millions of dollars of 1MDB money had gone missing in complex overseas transactions that have never been fully explained.
The Wall Street Journal reported this month that Malaysian government investigators had discovered that nearly $700m had moved through government agencies, banks and companies linked to 1MDB before ending up in Najib's personal accounts.
Last week the home ministry suspended the publishing permits of The Edge Media Group for three months. The media group is known for its aggressive reporting on 1MDB.
The overseas Sarawak Report, an activist website that had also published a series of exposes, has also been blocked.