The militants are fighting in the mountains of the southwestern Kadamdzhai district in the Batken Region.
"They are using natural defenses, and putting up strong resistance," the service said, adding that military reinforcements hade been sent to the scene.
Commenting on reports that two militants had already been seized, the source only said the information was being verified. However, the service said one gunman had been killed and identified as a Tajik national.
Kyrgyz border guards said earlier that a group of six gunmen had attacked the Lyakkan border post in Tajikistan's northern Isfarin district, killing two border guards, injuring a customs officer, and seizing 19 automatic rifles, a machinegun and cartridges.
The gunmen headed to Kyrgyzstan's Batken region in the south of the Fergana Valley, about 700 kilometers (440 miles) southwest of the capital, Bishkek. They are reported to have the hijacked a Mercedes Benz, killing the driver, after their Opel overturned.
Along the way to Osh, 100 kilometers (over 62 miles) from Batken, they attacked Kyrgyzstan's Putlon border post, killing one and wounding another border guard.
Kyrgyz border guards also said the same group had attacked a mobile border post and that two locals had been killed in the ensuing shootout.
Witnesses in Kyrgyzstan said the gunmen were traveling in a Mercedes and Opel. Then they left the Mercedes with 17 automatic rifles in it and walked in the direction of mountains.
Kyrgyzstan, Tajikistan, and Uzbekistan, which share the Fergana Valley, all have sent troops into the area.
The countries have seen a rise in Islamist extremism and popular unrests fueled by economic difficulties following the breakup of the Soviet Union.
Uzbek authorities blamed the May 2005 uprising in Andijan, on the border with Kyrgyzstan, on Islamic extremist groups linked to international terrorist organizations. Official statistics said 187 people died in the ensuing violence, but human rights groups said about 700 civilians died as a result of a violent crackdown by security forces and police.
