The special forces sergeant major has been under investigation since 2017.
According to reports he was hiding weapons at his home in Nordsachsen in the eastern state of Saxony.
The Dresden Prosecutor General said that a “large amount of evidence” was confiscated during the police raid.
"So far, ammunition, explosives and weapons have been made safe. The soldier involved has been arrested and is currently being questioned by the authorities concerned," said defence officials as quoted by the BBC.
The soldier is suspected of violating Germany’s Weapons Control Act.