In a 7-2 ruling, the Oklahoma Supreme Court said the monument violates the state's constitutional ban on using public property to benefit a religion.
The Ten Commandments are "obviously religious in nature and are an integral part of the Jewish and Christian faiths," the court said, according to the Associated Press.
The ruling overturns a decision by a district court judge who determined the monument could stay at the Capitol building in Oklahoma City.
A Republican legislator paid $10,000 of his own money to construct the 6-foot-tall granite monument in 2012. Some Republican lawmakers called for the impeachment of the justices who said the monument must be removed, the AP reported.
Attorney General Scott Pruitt had argued that the monument was historical in nature and nearly identical to a Texas monument that was found constitutional by the US Supreme Court.
"Quite simply, the Oklahoma Supreme Court got it wrong," Pruitt said in a statement. "The court completely ignored the profound historical impact of the Ten Commandments on the foundation of Western law."
Pruitt said the monument will remain intact while the court considers rehearing the case, as his office will request. He also suggested the provision in the Oklahoma Constitution that prohibits the use of public money for religious purposes may need to be repealed, the AP reported.
Ryan Kiesel, of the American Civil Liberties Union of Oklahoma, who represented the plaintiffs in the case, criticized Pruitt's suggestion and the calls for impeachment.
"I think the idea that you go about amending the constitution every time you lose a court battle is a dangerous precedent for anyone to engage in, but in particular for the state’s highest attorney to do so," Kiesel told the AP.
"And the calls for impeachment represent a fundamental misunderstanding of how an independent judiciary functions within our system of democratic government."
Other groups who have asked to build monuments on the Capitol grounds include a group that wants to erect a 7-foot-tall statue of Satan as Baphomet, a goat-headed figure with horns, wings and a long beard.
A Hindu leader in Nevada, an animal rights group, and the satirical Church of the Flying Spaghetti Monster also have made requests, the AP reported.
In October, a man smashed into and damaged the statue with his car. He was admitted to a hospital for mental health treatment, and formal charges were never filed. A new monument was put up in January.
