"Shinzo, why not join the North Atlantic Treaty Organization?" Merkel asked Abe.
"I can convince British Prime Minister Cameron and French President Hollande," she added.
The Yomiuri Shimbun reports that a surprised Abe replied, "Maybe in the future," in the knowledge that Japan is unlikely to join NATO.
"If we join now, our negotiations with Russia will stop," Abe explained, referring to talks between Japan and Russia regarding some of the Kuril Islands, a volcanic archipelago bordering the Sea of Okhotsk and the Pacific Ocean.
However, Japan claims four of the southern Kuril Islands are its territory, based on an 1855 trade treaty with Russia, and the notion that the four islands are not part of the Kuril Islands archipelago that Japan renounced sovereignty over in the 1951 treaty.
As a result of the territorial dispute Japan and the USSR, and consequently Russia, have never signed a permanent peace treaty since the end of the Second World War.