"I think every New Zealander would have an expectation that this individual should not be able to share his hateful message from behind bars", Ardern was quoted as saying by the New Zealand Herald daily.
The hand-written letter was reportedly posted on the 4chan imageboard. In it, Brenton Tarrant, who faces charges for killing 51 people in two mosques in March, writes about who inspired his political and social views.
The letter is reported to be addressed to a person named Alan who is said to live in Russia. The 28-year-old also discussed his last trip to Russia.
Inmates in New Zealand are allowed to send and receive letters but they can be confiscated if contents are deemed inappropriate. Corrections Minister Kelvin Davis said they had never had to deal with a terror suspect and suggested changing these rules.
The court ruled to keep Tarrant in custody until August 17 when the case review hearing is scheduled to be held.
In March, 51 people were killed as a result of an attack by an Australian man, Brenton Tarrant, who opened fire at two mosques in the city of Christchurch, New Zealand. Prior to the shooting, he wrote a 73-page "manifesto" about his plans.
The charges against Tarrant include 51 counts of murder and 40 counts of attempted murder, as well as engaging in a terrorist attack.
The New Zealand government subsequently passed a gun reform bill outlawing military-style semi-automatic weapons and assault rifles.