However, senior ANC leaders and most analysts believed the president would survive the vote given the party's large parliamentary majority.
Malema labelled Zuma as the "most corrupt individual in the country".
The no-confidence motion needs 201 votes to succeed, or a simple majority of parliament's 400 seats.
Even though the ballot was held in secret, the opposition was not able to persuade enough members of Zuma's ruling African National Congress to side with them.
If the motion is successful, Zuma along with 72 cabinet ministers would be forced to step down immediately, prompting the speaker to act as interim president for 30 days until the national assembly appoints a new president and cabinet.
Speaking afterwards, Mr. Zuma said he had come to thank his supporters and "those in parliament who had voted correctly".
The African National Congress (ANC) and opposition parties were at loggerheads before proceeding with the vote.
South Africa's Members of Parliament cast their votes in a secret ballot that seeks to remove President Jacob Zuma from his poistion.
"As you can see, thousands of people have reached the end of their tether in terms of what is happening in our attractive country, our attractive, diverse country that we should enjoy but we can't enjoy because millions of our people are without jobs", said one protester, Johnnie Jacobs.
If the ruling African National Congress party, which has the clear majority in parliament, can not agree on a candidate in that time, new national elections would be held.
"The ANC may have won in the no-confidence motion in parliament yesterday, but it has lost the confidence of the country", Maimane said.
DA chief whip John Steenhuisen added, "We are the official opposition in South Africa".
Opposition parties also planned a march in Cape Town to further push for Zuma's removal.
"I have faced seven kinds of vote of no confidence".
United Democratic Movement leader Bantu Holomisa said his party will join the EFF in court to impeach the president. Zuma's party has bailed him out from at least five previous attempts to remove him.
"Mbete's decision was made knowing that Zuma will be secure", Darias Jonker, of the New York-based Eurasia political analysis consultancy, said.