"These forces should be withdrawn, and both Russia and Georgia should engage quickly in a high-level and open dialogue to de-escalate tensions," he said in a statement on Tuesday.
The unarmed troops were deployed to the region on May 31 to repair rail tracks "fully in line with Russian-Georgian agreements," the Russian Foreign Ministry said.
"This deployment is clearly in contravention of Georgia's sovereignty and territorial integrity," said the NATO head however.
The presidents of Russia and Georgia discussed the issue over the telephone earlier Tuesday.
"Russia's aid in repairing railroad tracks in Abkhazia using its railroad troops was discussed. The necessary clarifications were provided on the issue," the Kremlin press service said, adding that Dmitry Medvedev and Mikheil Saakashvili had agreed to resume discussions on the sidelines of an economic forum in St. Petersburg due on June 6-8.
Tensions between Russia and Georgia have been consistently strained since Western-leaning President Mikheil Saakashvili came to power in the South Caucasus country in early 2004.
The long-running row over Georgian breakaway regions, along with Tbilisi's plans to join NATO, have been major factors behind the dispute. In a recent development, Georgia accused Russia of shooting down an unmanned reconnaissance plane over Abkhazia on April 20. Moscow denies involvement in the incident.
A UN report last Monday backed up Georgia's version of events, but criticized the country for carrying out surveillance flights in breach of the terms of a ceasefire deal that ended an armed conflict in the early 1990s.
