The incident took place on the premises of the 18th century Palace of Shaki Khans, which is a part of the UNESCO World Heritage.
"As a result of the incident, 19 people sought medical help. Two of them were hospitalized, the rest received first aid, and some of them are still being treated," the ministry said.
The Health Ministry added in a separated statement that four of the injured were in a serious condition, and they would be transported to the capital of Baku if necessary.
The ministry noted that there were two Russian citizens and five Syrian nationals among those injured.
Azerbaijani authorities said that President Ilham Aliyev had taken the incident under personal control.