The party lost in nine cities including the capital of Taipei and another major city — Taichung.
President Ma Ying-jeou promised to make changes and reshuffle the cabinet.
"I must express apologies to the Nationalist Party and its supporters for making everyone disappointed," AP quotes Ma as saying at a news conference. "I've received the message people have sent via these elections. It's my responsibility and I will quickly offer a party reform plan to address everyone's demands. I won't avoid responsibility."
Beijing has claimed sovereignty over Taiwan ever since the Chinese civil war of the 1940s, but after taking office in 2008, Ma has set aside the old disputes to ease tensions through talks.
A top Chinese official on Saturday night urged the people of Taiwan to protect those gains.
"We hope compatriots across the Strait will cherish hard-won fruits of cross-strait relations, and jointly safeguard and continue to push forward peaceful development of cross-strait relations," AP quotes Ma Xiaoguang, spokesman for the State Council Taiwan Affairs Office, as saying.
The heavy losses are now expected to make it tougher for Ma's Nationalists to hold onto the presidency in 2016.
